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Abstract. We give a new construction of a Hopf algebra defined first by Reading [Rea05] 
whose bases are indexed by objects belonging to the Baxter combinatorial family (i.e., Baxter 
permutations, pairs of twin binary trees, etc.). Our construction relies on the definition of the 
Baxter monoid, analog of the plactic monoid and the Sylvester monoid, and on a Robinson- 
Schensted-like correspondence and insertion algorithm. Indeed, the Baxter monoid leads to 
the definition of a lattice structure over pairs of twin binary trees and the definition of a 
Hopf algebra. The algebraic properties of this Hopf algebra are studied and among other, 
multiplicative bases are provided, and freeness and self-duality proved. 
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1. Introduction 

In recent years, many combinatorial Hopf algebras, whose bases are indexed by combina- 
torial objects, have been intensively studied. For example, the Malvenuto-Reutenauer Hopf 
algebra FQSym of Free quasi-symmetric functions [MR95, DHT02] has bases indexed by per- 
mutations. This Hopf algebra admits several Hopf subalgebras: The Hopf algebra of Free sym- 
metric functions FSym [PR95,DHT02], whose bases are indexed by standard Young tableaux, 
the Hopf algebra Bell [Rcy07] whose bases are indexed by set partitions, the Loday-Ronco Hopf 
algebra PBT [LR98, HNT05] whose bases are indexed by planar binary trees, and the Hopf 
algebra Sym of non-commutative symmetric functions [GKL+94] whose bases are indexed by 
integer compositions. A unifying approach to construct all these structures relies on a defini- 
tion of a congruence on words leading to the definition of monoids on combinatorial objects. 
Indeed, FSym is directly obtained from the plactic monoid [LS81, DHT02, Lot02], Bell from 
the Bell monoid [Rey07], PBT from the Sylvester monoid [HNT02, HNT05], and Sym from 
the hypoplactic monoid [KT97, Nov98] . The richness of these constructions relies on the fact 
that, in addition to constructing Hopf algebras, the definition of such monoids often brings par- 
tial orders, combinatorial algorithms and Robinson-Schensted-like algorithms, of independent 
interest. 

The Baxter combinatorial family admits various representations. The most famous of these 
are Baxter permutations [Bax64], which are permutations that avoid certain patterns, and pairs 
of twin binary trees [DG94]. This family also contains more exotic objects like quadrangula- 
tions [ABP04] and plane bipolar orientations [BBMF08]. In this paper, we propose to enrich the 
above collection of Hopf algebras by providing a plactic-like monoid, namely the Baxter monoid, 
leading to the construction of a Hopf algebra whose bases are indexed by objects belonging to 
this combinatorial family. 

In order to show examples of relations between lattice congruences [CS98] and Hopf alge- 
bras, Reading presented in [Rea05] a lattice congruence of the permutohedron whose equivalence 
classes are indexed by twisted Baxter permutations. These permutations were defined by a pat- 
tern avoidance property. This congruence is very natural: The meet of two lattice congruences 
of the permutohedron related to the construction of PBT is one starting point to build Sym; 
A natural question is to understand what happens when the join, instead of the meet, of these 
two lattice congruences is considered. Reading proved that his lattice congruence is precisely 
this last one, and that the minimal elements of its equivalence classes are twisted Baxter per- 
mutations. Besides, thanks to his theory, he gets for free a Hopf algebra whose bases are 
indexed by twisted Baxter permutations. Actually, twisted Baxter permutations are equinu- 
merous with Baxter permutations. Indeed, Law and Reading pointed out in [LR12] that the 
first proof occurred in unpublished notes of West. Hence, the Hopf algebra of Reading defined 
in [Rea05] can already be seen as a Hopf algebra on Baxter permutations, and our construction, 
considered as a different construction of the same Hopf algebra. Moreover, very recently, Law 
and Reading [LR12] detailed their construction of this Hopf algebra and studied some of its 
algebraic properties. 

We started independently the study of Baxter objects in a different way: We looked for a 
quotient of the free monoid analogous to the plactic and the Sylvester monoid. Surprisingly, 
the equivalence classes of permutations under our monoid congruence are the same as the 
equivalence classes of the lattice congruence of Law and Reading, and hence have the same 
by-products, as e.g., the Hopf algebra structure and the fact that each class contains both one 
twisted and one non-twisted Baxter permutation. However, even if both points of view lead to 
the same general theory, their paths are different and provide different ways of understanding 
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the construction, one centered on lattice theory, the other centered on combinatorics on words. 
Moreover, a large part of the results of each paper do not appear in the other as, in our case, the 
Robinson-Schensted-like correspondence and its insertion algorithm, the polynomial realization, 
the bidendriform bialgebra structure, the freeness, cofreeness, self-duality, primitive elements, 
and multiplicative bases of the Hopf algebra, and a few other combinatorial properties. 

We begin by recalling in Section 2 the preliminary notions about words, permutations, and 
pairs of twin binary trees used thereafter. In Section 3, we define the Baxter congruence. This 
congruence allows to define a quotient of the free monoid, the Baxter monoid, which has a 
number of properties required for the Hopf algebraic construction which follows. We show that 
the Baxter monoid is intimately linked to the Sylvester monoid and that the equivalence classes 
of the permutations under the Baxter congruence form intervals of the permutohedron. Next, 
in Section 4, we develop a Robinson-Schensted-like insertion algorithm that allows to decide if 
two words are equivalent according to the Baxter congruence. Given a word, this algorithm 
computes iteratively a pair of twin binary trees inserting one by one the letters of u. We give as 
well some algorithms to read the minimal, the maximal and the Baxter permutation of a Baxter 
equivalence class encoded by a pair of twin binary trees. We also show that each equivalence 
class of permutations under the Baxter congruence contains exactly one Baxter permutation. 
Section 5 is devoted to the study of some properties of the equivalence classes of permutations 
under the Baxter congruence. This leads to the definition of a lattice structure on pairs of 
twin binary trees, very similar to the Tamari lattice [Tam62, Knu06] since covering relations 
can be expressed by binary tree rotations. We introduce in this section twin Tamari diagrams 
that are objects in bijection with pairs of twin binary trees and offer a simple way to test 
comparisons in this lattice. Finally, in Section 6, we start by recalling some basic facts about 
the Hopf algebra of Free quasi-symmetric functions FQSym, and then give our construction of 
the Hopf algebra Baxter and study it. Using the polynomial realization of FQSym, we deduce 
a polynomial realization of Baxter. Using the order structure on pairs of twin binary trees 
defined in the above section, we describe its product as an interval of this order. Moreover, 
we prove that this Hopf algebra is free as an algebra by constructing two multiplicative bases, 
and introduce two operators on pairs of twin binary trees, analogous to the operators over and 
under of Loday-Ronco on binary trees [LR02]. Using the results of Foissy on bidendriform 
bialgebras [Foi07], we show that this Hopf algebra is also self-dual and that the Lie algebra of 
its primitive elements is free. We conclude by explaining some morphism with other known 
Hopf subalgebras of FQSym. 

This paper is an extended version of [Girl I]. It contains all proofs and Sections 4 and 6 have 
new results. 

Acknowledgments. The author would like to thank Florcnt Hivert and Jean-Christophe Novelli 
for their advice and help during all stages of the preparation of this paper. The computations 
of this work have been done with the open-source mathematical software Sage [S + 1 1] . 

2. Preliminaries 

2.1. Words, definitions and notations. In the sequel, A := {ai < 02 < • • • } is a totally 
ordered infinite alphabet and A* is the free monoid generated by A. Let u £ A*. We shall 
denote by \u\ the length of u and by e the word of length 0. The largest (resp. smallest) 
letter of u is denoted by max(u) (resp. rnin(u)). The evaluation ev(tt) of the word u is the non- 
negative integer vector such that its i-th entry is the number of occurrences of the letter ai in u. 
It is convenient to denote by Alph(u) := {ui : 1 < i ^ |u|} the smallest alphabet on which u is 
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defined. We say that is an inversion of u if i < j and ui > Uj. Additionally, i is descent 
of u if (i, i + 1) is an inversion of u. 

Let us now recall some classical operations on words. We shall denote by u~ := u\ u \ . . . u\ 
the mirror image of u and by u\g the restriction of u on the alphabet 5 C A, that is the longest 
subword of u such that Alph(u) C S. Let u e A*. The shuffle product LU is recursively defined 
on the linear span of words 1>{A) by 



(2.1) u LU v := < 



u if t> = e, 

v if u = e, 

a(w' LU bi/) + b(au' LU t/) otherwise, where u = au', « = bv', and a,be A 



For example, 

aia 2 LU a 2 ai = aia 2 a 2 ai + aia2a 2 ai + aia2aia 2 + a 2 aia 2 ai + a 2 aiaia 2 + a 2 aiaia 2 , 

(2-2) 

= 01020102 + 2 01020201 + 2a2aiaia2 + 02010201. 

Let A # := {af > af > ■ ■ ■} be the alphabet A on which the order relation has been reversed. 
The Schiitzenberger transformation # is defined on words by 



(2.3) m # = (ttitt 2 



:=U \u\--- U 2 u i ■ 



For example, (050301010502)* = afafafafafaf. Note that by setting := A, the 

transformation # becomes an involution on words. 



2.2. Permutations, definitions and notations. Denote by & n the set of permutations of 
size n and by 6 the set of all permutations. One can see a permutation of size n as a word 
without repetition of length n on the first letters of A. We shall call i a recoil of a e & n 
if (i, i + 1) is an inversion of cr -1 . By convention, n also is a recoil of o\ 

The (right) permutohedron order is the partial order defined on & n where a is covered 
by v if a = u ab v and v — ub&v where a < b G A, and u and v are words. Recall that one has 
a ^ if and only if any inversion of cr _1 also is an inversion of v^ 1 . 

Let a, v e 6. The permutation / v is obtained by concatenating ct and the letters of v 
incremented by |<rj; In the same way, the permutation o\v is obtained by concatenating the 
letters of v incremented by \a\ and a. For example, 

(2.4) 312/2314 = 3125647 and 312\ 2314 = 5647312. 

A permutation is connected if a = v / it implies v = a or 7r = a. Similarly, a is anti- connected 
if er~ is connected. The shifted shuffle product CD of two permutations is defined by 

(2.5) a CD v := a LU (z/i + |er| . . . V|„| + H) . 
For example, 

(2.6) 12 CD 21 = 12 LU 43 = 1243 + 1423 + 1432 + 4123 + 4132 + 4312. 

The standardized word std(u) of u € A* is the unique permutation of size |u| having the same 
inversions as u. For example, Std(a3aia402a5a7a4a2a3) = 416289735. 
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2.3. Binary trees, definitions and notations. We call binary tree any complete rooted 
planar binary tree. Recall that a binary tree T is either a leaf (also called empty tree) denoted 
by _L, or a node that is attached through two edges to two binary trees, called respectively 
the left subtree and the right subtree of T. Let BT n be the set of binary trees with n nodes 
and BT be the set of all binary trees. We use in the sequel the standard terminology (i.e., 
child, ancestor ; path, etc.) about binary trees [AU94]. In our graphical representations, nodes 
are represented by circles O, leaves by squares n, edges by segments / or \, and arbitrary 
subtrees by big squares like Q 

2.3.1. The Tamari order. The Tamari order [Tam62, Knu06] is the partial order defined 
on BT n where To £ BT n is covered by T\ £ BT n if it is possible to obtain T\ by performing a 
right rotation into Tq (see Figure 1). One has Tq Ti if and only if starting from Tq, it is 




Figure 1. The right rotation of root y. 



possible to obtain T\ by performing some right rotations. 

2.3.2. Operations on binary trees. If L and R are binary trees, denote by L A R the binary tree 
which has L as left subtree and R as right subtree. Similarly, if L and R are A-labeled binary 
trees, denote by LA&R the A-labeled binary tree which has L as left subtree, R as right subtree 
and a root labeled by a £ A. 

Let Tq,Ti £ BT. The binary tree To /T\ is obtained by grafting Tq from its root on the 
leftmost leaf of T\ ; In the same way, the binary tree To \ T± is obtained by grafting T\ from its 
root on the rightmost leaf of To . 

For example, for 

(2.7) T :=^°X and T 1 := P '"^ ] 



we have 




2.3.3. Binary search trees, increasing, and decreasing binary trees. An ^-labeled binary tree T 
is a right (resp. left) binary search tree if for any node x labeled by b, each label a of a node 
in the left subtree of x and each label c of a node in the right subtree of x, the inequality 
a ^ b < c (resp. a < b ^ c) holds. 

A binary tree T £ BT n is an increasing (resp. decreasing) binary tree if it is bijectively 
labeled on {1, . . . , n} and, for any node x of T, if y is a child of x, then the label of y is greater 
(resp. smaller) than the label of x. 
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The shape sh(T) of an /1-labeled binary tree T is the unlabeled binary tree obtained by 
forgetting its labels. 

2.3.4. Inorder traversal. The inorder traversal of a binary tree T consists in recursively visiting 
its left subtree, then its root, and finally its right subtree (see Figure 2). We shall say that a 




Figure 2. The sequence (a, 6, c, d, e, /, g, h) is the sequence of all nodes of this 
binary tree visited by the inorder traversal. 

node x is the i-th node of T if x is the i-th visited node by the inorder traversal of T. In the 
same way, a leaf y is the j-th leaf of T if y is the j-th visited leaf by the inorder traversal of T. 
We also say that i is the index of x and j is the index of y. If T is labeled, its inorder reading 
is the word u\ . . . iti u i such that for any 1 ^ i ^ Ui is the label of the i-th node of T. Note 
that when T is a right (or left) binary search tree, its inorder reading is a nondecreasing word. 

2.3.5. The canopy of binary trees. The canopy (see [LR98] and [Vie04]) cnp(T) of a binary 
tree T is the word on the alphabet {0, 1} obtained by browsing the leaves of T from left to 
right except the first and the last one, writing if the considered leaf is oriented to the right, 1 
otherwise (see Figure 3). Note that the orientation of the leaves in a binary tree is determined 
only by its nodes so that we can omit to draw the leaves in our graphical representations. 




o o u a i o l 
1 o 



Figure 3. The canopy of this binary tree is 0100101. 
2.4. Baxter permutations and pairs of twin binary trees. 

2.4.1. Baxter permutations. A permutation a is a Baxter permutation if for any subword u := 
U1U2U3U4 of a such that the letters ui and U3 are adjacent in a, std(tt) ^ {2413, 3142}. In other 
words, a is a Baxter permutation if it avoids the generalized permutation patterns 2 — 41 — 3 and 
3—14 — 2 (see [BS00] for an introduction on generalized permutation patterns). For example, 
42173856 is not a Baxter permutation; On the other hand 436975128, is a Baxter permutation. 
Let us denote by 6^ the set of Baxter permutations of size n and by 6 B the set of all Baxter 
permutations. 

2.4.2. Pairs of twin binary trees. A pair of twin binary trees (Tl,Tr) is made of two binary 
trees Tl , Tr 6 BT n such that the canopies of Tl and Tr are complementary, that is 

(2.10) cnp(T L )i ^ cnp(T fl )i for all 1 < i < n - 1 

(see Figure 4). 

Denote by TBT n the set of pairs of twin binary trees where each binary tree has n nodes 
and by TBT the set of all pairs of twin binary trees. 

An A- labeled pair of twin binary trees (Tl,Tr) is a pair of twin binary search trees if Tl 
(resp. Tr) is an A-labeled left (resp. right) binary search tree and Tl and Tr have the same 
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Figure 4. A pair of twin binary trees. 

inorder reading. The shape sh(J) of an ^4-labeled pair of twin binary trees J :— (Tj,,Tr) is the 
unlabeled pair of twin binary trees (sh(Tj,), sIi(Tr)). 

In [DG94], Dulucq and Guibert have highlighted a bijection between Baxter permutations 
and unlabeled pairs of twin binary trees. In the sequel, we shall make use of a very similar 
bijection. 

3. The Baxter monoid 

3.1. Definition and first properties. Recall that an equivalence relation = dchned on A* 
is a congruence if for all u,u',v,v' G A*, u = v! and v = v' imply uv = u'v'. Note that 
the quotient A* /= of A* by a congruence = is naturally a monoid. Indeed, by denoting by 
t : A* — > A* /= the canonical projection, the set A* /= is endowed with a product ■ defined by 
u ■ v := t(uv) for all m, v G A* /= where u and v are any words such that t(u) = u and t(v) = v. 

Definition 3.1. The Baxter monoid is the quotient of the free monoid A* by the congru- 
ence =b that is the reflexive and transitive closure of the Baxter adjacency relations 
and defined for u, v £ A* and a, b, c, d G A by 

(3.1) caadub^B cwdaub where a ^ b < c ^ d, 

(3.2) b u da v c ^b b u ad v c where a < b ^ c < d. 

For example, the =b -equivalence class of 2415253 (see Figure 5) is 

(3.3) {2142553, 2145253, 2145523, 2412553, 2415253, 2415523, 2451253, 2451523, 2455123}. 



2142553 



2145253 



2145523 



/ \ 

!53 24] 

/ \ / 

i23 2415253 

\ / \ 

i23 24f 

\ / 

2451523 

I 

2455123 



2412553 



2415523 



2451253 



2153674 



2156374 



2156734 



/ \ 

!74 25: 

/ \ / 

7 34 2516374 

\ / \ 

'34 25( 

\ / 

2561734 

I 

2567134 



2513674 



2516734 



2561374 



Figure 5. The Baxter equivalence class of the word u := 2415253 and of the 
permutation 2516374 = std(u). Edges represent Baxter adjacency relations. 

Note that if the Baxter congruence is applied on words without repetition, the two Baxter 
adjacency relations ^b and ^b can be replaced by the only adjacency relation ^b defined 
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for u,v E A* and a, b, b',d e A by 

(3.4) b u ad v b' b u da v b' where a<b,b'<d. 

3.1.1. Compatibility with the destandardization process. A monoid A* /= is compatible with the 
destandardization process if for all u, w G A*, u = w if and only if std(u) = std(w) and ev(w) = 
ev(w). 

Proposition 3.2. The Baxter monoid is compatible with the destandardization process. 

Proof. It is enough to check the property on adjacency relations. Let u,v G A*. Assume 
u v. We have 

(3.5) u = xcy&dzbt and v = xcyd&zbt 

for some letters a ^ b < c ^ d and words x, y, z, and t. Since ^b acts by permuting letters, 
we have ev(u) = ev(v). Moreover, the letters a', b', c' and d' of std(u) respectively at the same 
positions as the letters a, b, c and d of u satisfy a' < b' < c' < d' due to their relative positions 
into std(w) and the order relations between a, b, c and d. The same relations hold for the 
letters of std(w), showing that std(w) ^b std(v). The proof is analogous for the case u ^b v. 
Conversely, assume that v is a permutation of u and std(u) ^b std(u). We have 

(3.6) std(it) = xcyadzbi and std(u) = icj/dazbf 

for some letters a < b < c < d and words x, y, z, and t. The word u is a non-standardized 
version of std(w) so that the letters a', b', c' and d' of u respectively at the same positions as 
the letters a, b, c and d of std(u) satisfy a' ^ b' < c' ^ d' due to their relative positions into u 
and the order relations between a, b, c and d. The same relations hold for the letters of v, 
showing that u ^b v. The proof is analogous for the case std(u) ^b std(v). □ 

3.1.2. Compatibility with the restriction of alphabet intervals. A monoid A*/= is compatible 
with the restriction of alphabet intervals if for any interval / of A and for all u, v G A*, u = v 
implies u\i = v\j. 

Proposition 3.3. The Baxter monoid is compatible with the restriction of alphabet intervals. 

Proof. It is enough to check the property on adjacency relations. Moreover, by Proposition 3.2, 
it is enough to check the property for permutations. Let a, v G & n such that a ^-b v. We have 
a = xb y &d zb' t and v = xbydazb'i for some letters a < b,b' < d and words x, y, z, and t. 
Let / be an interval of {1, . . . , n} and R := I D {a, b, b', d}. If R = {a, b, b', d}, 

(3.7) ctij = xizbyn adzi/b tn and z/ij = xub j/i/da^i/b'ti/ 

so that o\i <=^b v\i- Otherwise, we have o\i — v\j and thus a\j =b v\i- Q 

3.1.3. Compatibility with the Schiitzenberger involution. A monoid A* /= is compatible with the 
Schiitzenberger involution if for all u, v G A* , u = v implies u# = i>*. 

Proposition 3.4. The Baxter monoid is compatible with the Schiitzenberger involution. 

Proof. It is enough to check the property on adjacency relations. Moreover, by Proposition 3.2, 
it is enough to check the property for permutations. Let ct, v G ©„ and assume that a ^b v. 
We have a — xbyaAzb't and v — xby d&zb' t for some letters a < b,b' < d and words x, y, 
z, and t. We have 

(3.8) a* = t*W* z# d*a* y*b* x* and v* = t* b'* z* a # d # y* b* x* . 

Since d# < b'#,b# < a#, we have er# <=^b v ■ D 
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3.2. Connection with the Sylvester monoid. The Sylvester monoid [HNT02, HNT05] is 
the quotient of the free monoid A* by the congruence =s that is the reflexive and transitive 
closure of the Sylvester adjacency relation ±=;g defined for u £ A* and a, b, c £ A by 

(3.9) aciib^g caub where a ^ b < c. 

In the same way, let us define the ^-Sylvester monoid, the quotient of A* by the congruence = s # 
that is the reflexive and transitive closure of the j^-sylvester adjacency relation defined 
for u G A* and a, b, c 6 A by 

(3.10) b«ac^ g #b«ca where a < b ^ c. 

Note that this adjacency relation is defined by taking the images by the Schutzenberger in- 
volution of the Sylvester adjacency relation. Indeed, for all u,v 6 A*, u = s # v if and only 
if u& =g u • 

In [HNT05], Hivert, Novelli and Thibon have shown that two words are Sylvester equivalent 
if and only if each gives the same right binary search tree by inserting their letters from right to 
left using the binary search tree insertion algorithm [AU94] . In our setting, we call this process 
the leaf insertion and it comes in two versions, depending on if the considered binary tree is a 
right or a left binary search tree: 

Algorithm: LeafInsertion. 

Input: An ^-labeled right (resp. left) binary search tree T, a letter a e A. 
Output: T after the leaf insertion of a. 

(1) If T =_L, return the one-node binary search tree labeled by a. 

(2) Let b be the label of the root of T. 

(3) If a sC b (resp. a < b): 

(a) Then, recursively leaf insert a into the left subtree of T. 

(b) Otherwise, recursively leaf insert a into the right subtree of T. 

End. 

For further reference, let us recall the following theorem due to Hivert, Novelli and Thi- 
bon [HNT05], restated in our setting and supplemented with a respective part: 

Theorem 3.5. Two words are =$ -equivalent (resp. = s # -equivalent) if and only if they give 
the same right (resp. left) binary search tree by inserting their letters from right to left (resp. 
left to right). 

In other words, any A-labeled right (resp. left) binary search tree encodes a Sylvester (resp. 
#-sylvester) equivalence class of words of A* , and conversely. 

Let us explain the respective part of Theorem 3.5. It follows from (3.10) that encoding 
the = s # -equivalence class of a word u is equivalent to encoding the =s -equivalence class 
of vfr . For this, simply insert u from left to right by considering that the reversed order 
relation holds between its letters. In this way, we obtain a binary tree such that for any node x 
labeled by a letter b, all labels a of the nodes of the left subtree of x, and all labels c of the 
nodes of the right subtree of x, the inequality a ^ b > c holds. This binary tree is obviously 
not a left binary search tree. Nevertheless, a left binary search tree can be obtained from it 
after swapping, for each node, its left and right subtree recursively. One can prove by induction 
on |u| that this left binary search tree is the one that LeafInsertion constructs by inserting 
the letters of u from left to right and hence, this remark explains the difference of treatment 
between right and left binary search trees for the instruction (3) of LeafInsertion. 
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Lemma 3.6. Let u := lacy and v :— xca.y be two words such that x and y are two words, 
a < c are two letters, and u =s v. Then, u v- 

Proof. Follows from Theorem 3.5: Since u and v give the same right binary search tree T by 
inserting these from right to left, the node labeled by a and the node labeled by c in T cannot be 
ancestor one of the other. That implies that there exists a node labeled by a letter b, common 
ancestor of both nodes labeled by a and c such that a ^ b < c. Thus, u v. □ 

Lemma 3.6 also proves that the -adjacency relations of any equivalence class C of ©„/ = s 
are exactly the covering relations of the permutohedron restricted to the elements of C. Note 
that it is also the case for the -adjacency relations. 

The Baxter monoid, the Sylvester monoid and the ^-Sylvester monoid are related in the 
following way. 

Proposition 3.7. Let u,v € A* . Then, u =b v if and only if u =g v and u = s # v. 

Proof. Once more, it is enough to check the property on adjacency relations. Moreover, by 

Proposition 3.2, it is enough to check the property for permutations. Let cr, v £ & n and assume 
that a v. We have a — xhyaAzh't and v — xbydazb' t for some letters a < b, b' < d and 
words x, y, z, and t. The presence of the letters a, d and b' with a < b' < d ensures that a v. 
Besides, the presence of the letters b, a and d with a < b < d ensures that cr v - 

Since the Sylvester and the ^-Sylvester monoids are compatible with the destandard- 
ization process [HNT05], it is enough to check the property for permutations. Let cr, v e 
6„ such that o=$v and a = s # v. Set r :~ inf ^ p {cr, v}. Since the permutohedron is a 
lattice, r is well-defined, and since the equivalence classes of permutations under the =s 
and = s # congruences are intervals of the permutohedron [HNT05] , we have a =s t =s v and 
a = s # t = g # v. Moreover, by Lemma 3.6, and again since that the equivalence classes of per- 
mutations under the =g and the = s # congruences are intervals of the permutohedron, for 
each saturated chains r cr' ■ • • ^p a and r ^p v' ^p • • • ^p v, there are sequences of ad- 
jacency relations t °~' 1 • ■ cr, T ^s* °~' ' ' ' °- T v ' • ■ ■ v an d 
t v' • • • v - Hence, r =b c and r =b v, implying a =b v. □ 

Proposition 3.7 shows that the =b -equivalence classes are the intersection of =s -equivalence 
classes and = s # -equivalence classes. 

By the characterization of the =b -equivalence classes provided by Proposition 3.7, restrict- 
ing the Baxter congruence on permutations, we have the following property: 

Proposition 3.8. For any n ^ 0, each equivalence class of 6 n / = B is an interval of the 
permutohedron. 

Proof. By Proposition 3.7, the =b -equivalence classes are the intersection of the =s and 
the = s # -equivalence classes. Moreover, the permutations under the =g and the = s # equiv- 
alence relations are intervals of the permutohedron [HNT05] . The proposition comes from the 
fact that the intersection of two lattice intervals is also an interval and that the permutohedron 
is a lattice. □ 

Lemma 3.9. Let u := xady and v := xday such that x and y are two words, a < d are two 
letters, and u =b V. Then, u ^b v or u ^b v. 

Proof. By Proposition 3.7, since u =b v, we have u =g v and thus by Lemma 3.6 we have u v , 
implying the existence of a letter b' in the factor y satisfying a ^ b' < d. In the same way, we 
also have u = s # v and thus u v, hence the existence of a letter b in the factor x satisfying 
a < b ^ d. That proves that u and v are ^b or ^b -adjacent. □ 
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Lemma 3.9 is the analog, in the case of the Baxter congruence, of Lemma 3.G and also proves 
that the and -adjacency relations of any equivalence class C of & n / = B are exactly 
the covering relations of the permutohedron restricted to the elements of C . 

3.3. Connection with the 3-recoil monoid. If a and c are two letters of A, denote by c — a 
the cardinality of the set {b G A : a < b ^ c}. In [NRT11], Novelli, Reutenauer and Thibon 
defined for any k ^ the congruence =r(*o . This congruence is the reflexive and transitive 
closure of the k-recoil adjacency relation, defined for a, b £ A by 

(3.11) ab ^ R (fc> ba where b — a ^ k. 

The k-recoil monoid is the quotient of the free monoid A* by the congruence = R (fc> . Note 
that the congruence = R <2> restricted to permutations is nothing but the hypoplactic congru- 
ence [Nov98]. 

The Baxter monoid and the 3-recoil monoid are related in the following way. 

Proposition 3.10. Each = R (3> -equivalence class of permutations can be expressed as a union 
of some =b -equivalence classes. 

Proof. This amounts to prove that for all permutations a and v, if a =b v then a = R <3> v. It 
is enough to check this property on adjacency relations. Hence, assume that a ^b v. We have 
a = xbyadzWt and v — a; b ?/ da z b' i for some letters a < b,b' < d and words x, y, z, and t. 
Since a and v are permutations, b ^ b' and thus, we have a < b < b' < d or a < b' < b < d, 
implying that d — a ^ 3. Hence, a = R <3> v. □ 

Note that Proposition 3.10 is false for the congruence = R (4> since there are twenty- two equiv- 
alence classes of permutations of size 4 under the congruence =b but twenty-four under = R <4> . 
Conversely, note that = R <4) is not a refinement of =b since for any n ^ 5, the permutation 
l.n.n — 1 . . .2 is the only member of its =b -equivalence class but not of its = R <4) -equivalence 
class. 

Moreover, it is clear, by definition of = R (*o , that the = R <fc) -equivalence classes of permu- 
tations are union of = R (k+i) -equivalence classes. Hence, by Proposition 3.10, the hypoplactic 
equivalence classes of permutations are union of some =b -equivalence classes. 

4. A ROBINSON-SCHENSTED-LIKE ALGORITHM 

The goal of this section is to define an analog to the Robinson-Schensted algorithm for the 
Baxter monoid — see [LS81,Lot02] for the usual Robinson-Schensted insertion algorithm that 
associate to any word u its P-symbol, that is a Young tableau. 

The interest of the Baxter monoid in our context is that the equivalence classes of the 
permutations of size n under the Baxter congruence are equinumerous with unlabeled pairs of 
twin binary trees with n nodes, and thus, by the results of Dulucq and Guibert [DG94], also 
equinumerous with Baxter permutations of size n. We shall provide a proof of this property in 
this section, using our analog of the Robinson-Schensted algorithm. 

4.1. Principle of the algorithm. We describe here an algorithm testing if two words are 
equivalent according to the Baxter congruence. Given a word u G A*, it computes its Baxter 
P-symbol, that is an ^4-labeled pair (T^,Tfl) consisting in a left and a right binary search tree 
such that the nondecreasing rearrangement of u is the inorder reading of both Tl and Tr. 
It also computes its Baxter Q-symbol, that is a pair of twin binary trees (Sl,Sr) where Sl 
(resp. Sr) is an increasing (resp. decreasing) binary tree, such that the inorder reading of Sl 
and Sr are the same. Moreover, Tl and Sl have same shape, and so have Tr and Sr. 
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4.1.1. The Baxter P -symbol. 

Definition 4.1. The Baxter P-symbol (or simply P-symbol if the context is clear) of a word 
u G A* is the pair P(u) — (Tl,Tr) where Tl (resp. Tr) is the left (resp. right) binary search 
tree obtained by leaf inserting the letters of u from left to right (resp. right to left). 

Figure shows the P-symbol of u := 2415253. Before showing that the P-symbol of Defini- 
tion 4.1 can be used to decide if two words are equivalent under the Baxter congruence, let us 
give an intuitive explanation of its validity. 

Recall that, according to Proposition 3.7, to represent the Baxter equivalence class of a 
word u, one has to represent both the equivalence class of u under the =s congruence and the 
equivalence class of u under the = s # congruence. This is exactly what the Baxter P-symbol 
does since, for a word u, it computes a pair (Tl,Tr) where, by Theorem 3.5, Tl represents 
the = s # -equivalence class of u and Tr represents the =s -equivalence class of u. 

4.1.2. The Baxter Q-symbol. Let us first recall two algorithms. Let u be a word. Define incr(u), 
the increasing binary tree of u recursively by 

{_L if u = e, 

incr(w) A a incr(u>) where u — v&w, a = min(u), and a < min(w). 

In the same way, define the decreasing binary tree of u decr(u), by 

{_L if u = e, 

decr(w) Ab decr(u;) where u = vbw, b = max(u), and b > max(w). 

Definition 4.2. The Baxter Q-symbol (or simply Q-symbol if the context is clear) of a word 
u E A* is the pair Q(u) = (Sl, St) where 

(4.3) S L ■= incr(std(u) _1 ) and S R := deer (std^) -1 ) . 

Figure G shows the Q-symbol of u := 2415253, whose standardized word is 2516374, so that 
std(u)" 1 = 3157246. 




Figure 6. The P-symbol and the Q-symbol of u := 2415253. 



It is plain that given a word u, the Q-symbol of u allows, in addition with its P-symbol, to 
retrieve the original word. Indeed, if P(u) = (Tl,Tr) and Q(u) = (Sl,Sr), the pair (Tr,Sr) 
is the output of the Robinson-Schensted-like algorithm in the context of the Sylvester monoid, 
which is a bijection between words and pairs of such binary trees [HNT05]. Given (Tr, Sr), it 
amounts to reading the labels of Tr in the order of the corresponding labels in Sr. The same 
holds of the pair (Tl, Sl)- 
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4.2. Correctness of the insertion algorithm. 

Lemma 4.3. Let T be a non-empty binary tree and y be the i-th leaf ofT. If y is left- oriented, 
it is attached to the i-th node ofT. If y is right-oriented, it is attached to the i—l-st node ofT. 

Proof. We proceed by structural induction on the set of non-empty binary trees. If T is the 
one-node binary tree, the lemma is clearly satisfied. Otherwise, we have T — A A B. Let y be 
the i-th leaf of T and x be the node where y is attached. If y is also in A and A =_L, y is left- 
oriented and is attached to the root of T (that is the first node of T) and the lemma is satisfied. 
If y is in A and A ^_L, y is also the i-th leaf of A and a; is a node of A, so that the lemma 
follows by induction hypothesis on A. Otherwise, y is in B. If B =+, y is right-oriented and is 
attached to the root of T (that is the last node of T) and the lemma is satisfied. Otherwise, y 
is the i— (n+l)-st leaf of B where n is the number of nodes of A. Assume that the node x is 
the j-st node of T, then, x becomes the j— (n+l)-st node of B. Hence, the lemma follows by 
induction hypothesis on B. □ 

The following proposition is the key of our construction. 

Proposition 4.4. Let a be a permutation and T be the left binary search tree obtained by left 
leaf insertions of the letters of a, from left to right. Then, the i+l-st leaf ofT is right- oriented 
if and only if i is a recoil of a. 

Proof. Set a := i and c := i+1. Assume that a is a recoil of a. We have a — ucvaw for some 
words u, v, and w. Since no letter b of u and v satisfies a < b < c, the node of T labeled by c 
has a node labeled by a in its left subtree, itself having no right child and thus contributes, by 
Lemma 4.3, to a right-oriented leaf in position i + 1. 

Conversely, assume that a is not a recoil of a. We have a = u&vcw for some words u, v, 
and w. For the same reason as before, the node of T labeled by a has a node labeled by c in its 
right subtree, itself having no left child and thus contributes, by Lemma 4.3, to a left-oriented 
leaf in position i + 1. □ 

Figure 7 shows an example of application of Proposition 4.4. 



Figure 7. The binary search tree drawn with its leaves obtained by left leaf 
insertions of the letters of a := 4136275, from left to right. The recoils of a are 
2, 3, 5, and 7 and the 3-rd, 4-th, 6-th, and 8-th leaves of this binary tree are 
right-oriented. 

4.2.1. The P-symbol. 

Proposition 4.5. For any word u G A*, the P-symbol (Tl,Tr) of u is a pair of twin binary 
search trees — Tl (resp. Tr) is a left (resp. right) binary search tree, and the inorder reading of 
both Tl and Tr is the nondecreasing rearrangement of u. 

Proof. Note by definition of the LeafInsertion algorithm that Tl (resp. Tr) is a left (resp. 
right) binary search tree and the inorder reading of both Tl and Tr is the nondecreasing 
rearrangement of u. It is plain that the leaf insertion of u and std(it) from left to right (resp. 
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right to left) into left (resp. right) binary search trees give binary trees of same shape. That 
implies that we can consider that u =: a is a permutation. Proposition 4.4 implies that the 
canopies of Tl and Tr are complementary because i is a recoil of a if and only if i is not a 
recoil of er~. Thus, the shapes of Tl and Tr consist in a pair of twin binary trees. □ 

Theorem 4.6. Let u,v G A* . Then, u =b v if and only if P(u) = P(v). 

Proof. Assume u =b v. Then, by Proposition 3.7, u and v are =g and = s # -equivalent. Hence, 
by Theorem 3.5, u and v have the same Sylvester and ^-Sylvester P-symbol, so that P(u) — P(v). 

Conversely assume that P(u) = P(u) =: (Tl,Tr). Since the leaf insertion of both u and v 
from left to right gives Tl, we have, by Theorem 3.5, u =g# v. In addition, the leaf insertion 
of both u and v from right to left gives Tr, so that, by the just cited theorem, u =g v. By 
Proposition 3.7, we have u =b v. □ 

In the case of permutations, each =b -equivalence class can be encoded by an unlabeled pair 
of twin binary trees because there is one unique way to bijectively label a binary tree with n 
nodes on {1, . . . , n} such that it is a binary search tree. Hence, in the sequel, unlabeled pairs 
of twin binary search trees can be considered as labeled by a permutation, and conversely. 

4.2.2. The Q-symbol. Let us recall the following lemma of [HNT05], restated in our setting and 
supplemented with a respective part: 

Lemma 4.7. Let u be a word and a := std(u) . The right (resp. left) binary search tree 
obtained by inserting u from right to left (resp. from left to right) and decr(er) (resp. incr(cr),) 
have same shape. 

Proposition 4.8. For any word u G A*, the shape of the Q-symbol (Sl,Sr) of u is a pair of 
twin binary trees. Moreover, Sl is an increasing binary tree, Sr is a decreasing binary tree and 
their inorder reading is std(tt) . 

Proof. By definition of the Q-symbol, Sl and Sr are respectively the increasing and the de- 
creasing binary trees of a := std(u) . By Lemma 4.7, a binary tree with same shape as Sl 
(resp. Sr) can also be obtained by leaf insertions of the letters of er -1 from left to right (resp. 
right to left). Thus, by Proposition 4.4, the shape of (Sl,Sr) is a pair of twin binary trees. 
Moreover, by the definition of the algorithms incr and deer, we can prove by induction on the 
size of a that the binary trees Sl and Sr have both a as inorder reading. □ 

Theorem 4.9. The map u i — > (P(u),Q(u)) is a bisection between the elements of A* and the 
set formed by the pairs ((Tl,Tr), (Sl,Sr)) where 

(i) (Tl,Tr) is a pair of twin binary search trees — Tl (resp. Tr) is a left (resp. right) 
binary search tree, and Tl and Tr have both the same inorder reading; 

(ii) (Sl,Sr) is a pair of twin binary trees where Sl (resp. Sr) is an increasing (resp. 
decreasing) binary tree, and Sl and Sr have both the same inorder reading; 

(Hi) {Tl,Tr) and {Sl,Sr) have same shape. 

Proof. Let us first show that for any u G A*, the pair (P(u),Q(u)) satisfies the assertions of 
the theorem. Point (i) follows from Proposition 4.5. Point (ii) follows from Proposition 4.8. 
Moreover, by Lemma 4.7, Point (iii) checks out. Besides, as already mentioned, it is possible 
to reconstruct from the pair (P(w), Q(u)) the word u and such a word is unique. That shows 
that the correspondence is well-defined and injective. 

Conversely, assume that {(Tl,Tr),{Sl,Sr)) satisfies the three assertions of the theorem. 
According to [HNT02], there is a bijection between the elements of A* and the pairs (Tr, Sr) 
where Tr is a right binary search tree and Sr a decreasing binary tree of same shape. Let u 
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be the word in correspondence with (Tr,Sr). In the same way, there is a bijection between 
the elements of A* and the pairs (Tl,Sl) where Tl is a left binary search tree and Sl an 
increasing binary tree of same shape. Let v be the word in correspondence with (Tl, Sl). By 
hypothesis, Tl and Tr have both the same inorder reading, implying ev(u) = ev(w). In the 
same way, since Sl and Sr have both the same inorder reading, one has std(u) -1 = std(u) -1 . 
Hence, we have std(u) = std(w) and thus u = v. Note also that the pair (Tl,Sl) is entirely 
determined by the pair (Tr,Sr) and conversely. Now, again according to [HNT02], the pair 
(Tr, Sr) is the Sylvester P-symbol of u and the pair (Tl, Sr) is the #-sylvester P-symbol of u. 
Hence, the insertion of u gives the pair ((Tl,Tr), (Sl, Sr)), showing that the correspondence 
is also surjective. □ 

4.3. Distinguished permutations from a pair of twin binary trees. We present in this 
section some algorithms to read some distinguished permutations from a pair of twin binary 
search trees. Let us first start with a useful characterization of =b -equivalence classes. 

4.3.1. Baxter equivalence classes as linear extensions of posets. Let T be an v4-labeled binary 
tree. We shall denote by A(T) (resp. v(^)) the poset (N, where N := {1, . . . ,n}, n is the 
number of nodes of T, and ^ is defined, for i,j 6 N, by 

(4.4) i ^ j if the i-th node is an ancestor (resp. descendant) of the j-th node of T. 

If the sequence i\ . . . i n is a linear extension of A(T) (resp. Xj(T)), we shall also say that the 
word u\ . . . u n is a linear extension of A(T) (resp. v(^)) if f° r an y 1 ^ ^ ^ n i the label of the 
z^-th node of T is ui. 

The words of a Sylvester equivalence class encoded by a labeled right binary search tree T 
coincide with the linear extensions of v(^) ( see Note 4 of [HNT05]). Additionally, this also 
says that the words of a #-sylvester equivalence class encoded by a labeled left binary search 
tree T are exactly the linear extensions of A(T). One has a similar characterization of Baxter 
equivalence classes: 

Proposition 4.10. The words of a Baxter equivalence class encoded by a pair of twin binary 
search trees (Tl,Tr) coincide with the words that are both linear extensions of the posets A(Tl) 
and S/(Tr). 

Proof. Let u be a word belonging to the Baxter equivalence class encoded by (Tl,Tr). By 
Theorem 4.6, Tl (resp. Tr) can be obtained by leaf inserting u from left to right (resp. right 
to left). Hence, if i ^ j in A(T^) (resp. in \/(Tr)) then i is smaller than j as integers. Thus, u 
is a linear extension of both A(Tl) and \/(Tr). 

Assume now that a is a linear extension of A(T^) and \/(Tr) and let v be any word of 
the Baxter equivalence class encoded by (T L ,T R ). By Theorem 4.6, Tl (resp. Tr) can be 
obtained by leaf inserting v from left to right (resp. right to left). Note 4 of [HNT05] implies 
that u = s # v and u =s v. Hence, by Proposition 3.7, one has u =b v, showing that u also 
belongs to the Baxter equivalence class represented by (Tl,Tr). □ 

To illustrate Proposition 4.10, consider the following labeled pair of twin binary search trees, 



(4.5) 




1(5 



SAMUELE GIRAUDO 



The set of words that are linear extensions of A(Tl) and vC^fl) are (the highlighted permuta- 
tion is a Baxter permutation) 

{5214376, 5214736, 5217436, 5241376, 5241736, 

(4.6) 

v ' 5247136, 5271436, 5274136, 5721436, 5724136}, 

which is exactly the Baxter equivalence class encoded by (Il,Tr). 

Note that it is possible to represent the order relations induced by the posets A(Tl) and 
Vpit) in only one poset A(T£,)Uv(Tr), adding on A(T^) the order relations induced by v(Tr). 
For the previous example, we obtain the poset 



4.3.2. Extracting Baxter permutations. The following algorithm allows, given an ^4-labeled pair 
of twin binary search trees (Tl, Tr), to compute a word belonging to the =b -equivalence class 
encoded by (Tl,Tr). When (Tl,Tr) is labeled by a permutation, our algorithm coincides with 
the algorithm designed by Dulucq and Guibert to describe a bijection between pairs of twin 
binary trees and Baxter permutations [DG94]. Besides, since their algorithm always computes a 
Baxter permutation, our algorithm also returns a Baxter permutation when (Tl,Tr) is labeled 
by a permutation. 

Algorithm: ExtractBaxter. 

Input: An A- labeled pair of twin binary search trees (Tl,Tr). 

Output: A word belonging to the Baxter equivalence class encoded by (Tl,Tr). 

(1) Let u := e be the empty word. 

(2) While T L and T R ^_L: 

(a) Let a be the label of the root of Tl. 

(b) Let i be the index of root of Tl. 

(c) Set u := wa. 

(d) Let A (resp. B) be the left (resp. right) subtree of Tl- 

(e) If the z-th node of Tr is a left child in Tr: 

(i) Then, set T L := A/B. 

(ii) Otherwise, set T L := A\B. 

(f) Suppress the i-th node in Tr. 

(3) Return u. 



Figure 8 shows an execution of this algorithm. 

The results of Dulucq and Guibert [DG94] imply that ExtractBaxter terminates. The 
only thing to prove is that the computed word belongs to the =b -equivalence class encoded by 
the pair of twin binary search trees as input. For that, let us first prove the following lemma. 

Lemma 4.11. Let (T^,Tr) be a non-empty pair of twin binary trees. If the root of Tl is the 
i-th node ofT^, then, the i-th node ofT^ has no child. 

Proof. Assume that Tl = A A B. Note that if both A and B are empty, Tl and Tr are the 
one-node binary trees and the lemma is clearly satisfied. 



(4.7) 




End. 
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FIGURE 8. An execution of the algorithm ExtractBaxter on (T L , Tr). The 
computed Baxter permutation is 562134. 

If A ^_L, assume that the i-th node of Tr has a non-empty left subtree. That implies that 
the i-th leaf of Tr is not attached to its i-th node. Thus, by Lemma 4.3, the i-th leaf of Tr 
is attached to its z — 1-st node and is right-oriented. In Tl, the i-th leaf cannot be attached to 
its i-th node because A ^_L. Hence, by Lemma 4.3, the i-th leaf of Tl is also attached to its 
i — 1-st node and is right-oriented. Since T contains at least i nodes, there is at least i+1 leaves 
in T, implying that the i-th leaf is not the rightmost leaf of Tl and Tr, and thus (Tl,Tr) is 
not a pair of twin binary trees, contradicting the hypothesis. 

Assume now that the i-th node of Tr has a non-empty right subtree. That implies that the 
i+l-st leaf of Tr is not attached to its i-th node and thus, by Lemma 4.3, the i+l-st leaf of Tr 
is left-oriented. Moreover, since the i-th node of Tr has a non-empty right subtree and the i-th 
node of Tl is its root, the i-th node of Tl also has a non-empty right subtree. That implies 
that the i + l-st leaf of Tl is not attached to its i-th node and thus, by Lemma 4.3, the i+l-st 
leaf of Tr is also left-oriented. That contradicts that (Tl,Tr) is a pair of twin binary trees, 
and implies that the i-th node of Tr has no child. The case B is analogous. □ 

Proposition 4.12. For any A-labeled pair of twin binary search trees (Tl,Tr) as input, the 
algorithm ExtractBaxter computes a word belonging to the =b -equivalence class encoded 
by (Tl,Tr). Moreover, if (Tl,Tr) is labeled by a permutation, the computed word is a Baxter 
permutation. 

Proof. Let us prove by induction on n, that is the number of nodes of Tl and Tr, that if (Tl,Tr) 
is an A-labeled pair of twin binary search trees, then ExtractBaxter returns a word that 
is a linear extension of A(T^) and a linear extension of \/(Tr), i.e., by Proposition 4.10, a 
word belonging to the =b -equivalence class encoded by (Tl,Tr). This property clearly holds 
for n ^ 1. Now, assume that Tl = AA a B where a is the label of the root of Tl- By Lemma 4.11, 
if the root of Tl is its i-th node, the i-th node x of Tr has no child. Moreover, since Tl and Tr 
are binary search trees and labeled by a same word, their respective i-th nodes have the same 
label a. Moreover, the canopy of Tl is of the form vOlw where v :— cnp(^4) and w := cnp(B), 
and the canopy of Tr is of the form v'lOw' where v' (resp. w') is the complementary of v 
(resp. w) since that (Tl, Tr) is a pair of twin binary trees. We have now two cases whether x 
is a left of right child in Tr. 

If a; is a left child in Tr, the algorithm returns the word ait where u is the word obtained 
by applying the algorithm on {T' L ,T' R ) where T' L — A/ B and T' R is obtained from Tr by 
suppressing the node x. First, the canopy of T' L is of the form vOw and the canopy of T' R 
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is of the form v'lw'. Moreover, T' L and T' R are clearly still binary search trees. That implies 
that (T' Ll T'x) is a pair of twin binary search trees. By induction hypothesis and Proposition 4.10, 
the word u belongs to the =b -equivalence class encoded by (T^,T R ), and thus, an belongs 
to the =b -equivalence class encoded by (Tl,Tr) because au is a linear extension of A(T^) 
(resp. \/(Tr)) since u is a linear extension of both A(T^) and \/{T' R ). The case where a; is a 
right child in Tr is analogous. 

Finally, when (Tl,Tr) is labeled by a permutation, ExtractBaxter coincides with the 
algorithm of Dulucq and Guibert [DG94] and computes a Baxter permutation. □ 

The validity of ExtractBaxter implies the two following results. 

Theorem 4.13. For any n ^ 0, there is a bijection between the set of Baxter equivalence 
classes of words of length n and A-labeled pairs of twin binary search trees with n nodes. 

Proof. By Proposition 4.5 and Theorem 4.6, the P-symbol algorithm induces an injection be- 
tween the set of equivalence classes of & n / = B and the set of unlabeled pairs of twin binary 
trees. Moreover, by Proposition 4.12, the algorithm ExtractBaxter exhibits a surjection 
between these two sets. Hence, these two sets are in bijection. □ 

Theorem 4.13 implies in particular that the Baxter equivalence classes of permutations of 
size n are in bijection with pairs of twin binary trees labeled by a permutation (or equivalcntly 
with unlabeled pairs of twin binary trees) . 

Theorem 4.14. For any n ^ 0, each equivalence class of& n /= B contains exactly one Baxter 
permutation. 

Proof. Let C be an equivalence class of & n / = B • By Theorem 4.13, C can be represented by an 
unlabeled pair of twin binary trees J. By Proposition 4.12, the algorithm ExtractBaxter 
computes a permutation belonging to the =b -equivalence class encoded by J, showing that 
each =b -equivalence class of permutations contains at least one Baxter permutation. The 
theorem follows from the fact that Baxter permutations are equinumerous with unlabeled pairs 
of twin binary trees. □ 

4.3.3. Extracting minimal and maximal permutations. Reading defined in [Rca05] twisted Bax- 
ter permutations, that are the permutations avoiding the generalized permutation patterns 2 — 
41 — 3 and 3 — 41 — 2. These permutations are particular elements of Baxter classes of permu- 
tations: 

Proposition 4.15. Twisted Baxter permutations coincide with minimal elements of Baxter 
equivalence classes of permutations. 

Proof. First, note that by Proposition 3.8, every Baxter equivalence class of permutations has 
a minimal element. Assume that a is minimal of its =b -equivalence class of permutations. 
Then, it is not possible to perform any rewriting of the form 

(4.8) budaub'^bwadjjb', 

where a < b, b' < d are letters, and u and v are words. Hence, a avoids the patterns 2 — 41 — 3 
and 3 — 41 — 2, and is a twisted Baxter permutation. 

Conversely, if a is a twisted Baxter permutation, it avoids 2 — 41 — 3 and 3 — 41 — 2 and it 
is not possible to perform any rewriting — >, so that, by Proposition 3.8 and Lemma 3.9, it is 
minimal of its =b -equivalence class. □ 
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In a similar way, by calling anti-twisted Baxter permutation any permutation that avoids the 
generalized permutation patterns 2—14 — 3 and 3 — 14 — 2, an analogous proof to the one of 
Proposition 4.15 shows that anti-twisted Baxter permutations coincide with maximal elements 
of Baxter equivalence classes of permutations. 

Proposition 4.15 implies that twisted Baxter permutations, anti-twisted Baxter permuta- 
tions, and Baxter permutations are equinumerous since by Theorem 4.14 there is exactly one 
Baxter permutation by =b -equivalence class of permutations and by Proposition 3.8, there is 
also exactly one twisted (and one anti-twisted) Baxter permutation. This suggests among other 
that there exists a bijection sending a Baxter permutation to the twisted Baxter permutation 
of its =b -equivalence class. 

As pointed out by Law and Reading, West has shown first a bijection between Baxter per- 
mutations and twisted Baxter permutations using generating trees [BM03]. In our setting, as 
in the setting of Law and Reading [LR12], this bijection is the one preserving the classes. Here 
follows an algorithm to compute this bijection. 

Let us consider the following algorithm which allows, given an A-labeled pair of twin binary 
search trees (Tj,, Tr), to compute the minimal permutation for the lexicographic order belonging 
to the =b -equivalence class encoded by (Tl,Tr). 

Algorithm: ExtractMin. 

Input: An A-labelcd pair of twin binary search trees (Tl, Tr). 

Output: The minimal word for the lexicographic order of the class encoded by (Tl,Tr). 

(1) Let u := e be the empty word. 

(2) Let F :— Tl be a rooted forest. 

(3) While F is not empty and T R =/=±: 

(a) Let i be the smallest index such that the i-th node of F is a root and the i-th 
node of Tr has no child. 

(b) Let a be the label of the i-th node of T^. 

(c) Set u := 

(d) Suppress the i-th node of F and the i-th node of Tr. 

(4) Return u. 

End. 

Note that, by choosing in the instruction (3a) the greatest index instead of the smallest, 
the previous algorithm would compute the maximal word for the lexicographic order of the 
=b -equivalence class encoded by (Tj,,Tr). Let us call this variant ExtractMax. 

Figure 9 shows an example of application of ExtractMin. 

Proposition 4.16. For any A-labeled pair of twin binary search trees (Tl,Tr) as input, the al- 
gorithm ExtractMin (resp. ExtractMax ) computes the minimal (resp. maximal) word for 
the lexicographic order of the =b -equivalence class encoded by (Tl,Tr). Moreover, if (Tl,Tr) 
is labeled by a permutation, the computed word is the minimal (resp. maximal) permutation for 
the permutohedron order of its =b -equivalence class. 

Proof. The output u of the algorithm ExtractMin (resp. ExtractMax) is both a linear 
extension of A(T^) and a linear extension of vCTr)- That implies by Proposition 4.10 that u 
belongs to the =b -equivalence class encoded by the input pair of twin binary trees. Moreover, 
this algorithm terminates since by Theorem 4.14, each A-labeled pair of twin binary search 
trees (Tl, Tr) admits at least one word that is a common linear extension of A(Tl) and \/(Tr). 



20 SAMUELE GIRAUDO 




^ © © ^ ±± 

FIGURE 9. An execution of the algorithm ExtractMin on (T L ,T R ). The 
computed permutation is 521364 and it is minimal in its =b -equivalence class. 

The minimality (resp. maximality) for the lexicographic order of the computed word comes 
from the fact that at each step, the node that has the smallest (resp. greatest) label is chosen. 

Finally, since the lexicographic order is a linear extension of the permutohedron order, and 
by Proposition 3.8, since Baxter equivalence classes are intervals of the permutohedron, Ex- 
tractMin (resp. ExtractMax) returns the minimal (resp. maximal) permutation for the 
permutohedron order of its Baxter equivalence class. □ 

By Proposition 4.16 and using our Robinson-Schenstcd-likc algorithm, we can compute the 
bijection between Baxter permutations and twisted Baxter permutations in the following way: 
If a is a Baxter permutation, apply ExtractMin on P(ct) to obtain its corresponding twisted 
Baxter permutation. Conversely, if a is a twisted Baxter permutation, apply ExtractBaxter 
on P(cr) to obtain its corresponding Baxter permutation. 

In the same way, we can compute a bijection between Baxter permutations and anti-twisted 
Baxter permutations using ExtractMax instead of ExtractMin. Moreover, these algo- 
rithms give a bijection between twisted Baxter permutations and anti-twisted Baxter permu- 
tations: If a is a twisted (resp. anti-twisted) Baxter permutation, apply ExtractMax (resp. 
ExtractMin) on P(cr) to obtain its corresponding anti-twisted (resp. twisted) Baxter permu- 
tation. 

4.4. Definition and correctness of the iterative insertion algorithm. In what follows, 
we shall revise our P-symbol algorithm that we have presented in Section 4.1 to make it iterative. 
Indeed, we propose an insertion algorithm such that, for any word u such that P(u) = (Tl, Tr) 
and any letter a, the insertion of a into (?l,Tr) is the pair of twin binary trees P(ua). This, 
besides being in agreement with the usual Robinson-Schensted-likc algorithms, has the merit 
to allow to compute in the Baxter monoid. Indeed, this gives a simple way to compute the 
concatenation of two words u and v under the Baxter congruence simply by inserting the letters 
of the word uv into the pair (_L, _L). Note that one can compute the product of two pairs of twin 
binary trees (Tl, Tr) and (T L , Tjj) by computing a word u' that belongs to the =b -equivalence 
class of (T' L , Tfj) by applying the algorithm ExtractMin (or ExtractBaxter) with (T' L , T^) 
as input, and then, by inserting the letters of u' from left to right into (T£,Tr). 

4.4.1. Root insertion in binary search trees. Let T be an A-labeled right binary search tree 
and b a letter of A. The lower restricted binary tree of T compared to b, namely T^ b , is the 
right binary search tree uniquely made of the nodes xofT labeled by letters a satisfying a ^ b 
and such that for all nodes x and y of T^ b , if x is ancestor of y in T^ b , then x is also ancestor 
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of y in T. In the same way, we define the higher restricted binary tree of T compared to b, 
namely T> b (see Figure 10). 



Let T be an ^4-labeled right binary search tree and a a letter of A. The root insertion of a 
into T consists in modifying T so that the root of T is a new node labeled by a, its left subtree 
is T^ a and its right subtree is T >a . 

4.4.2. The iterative insertion algorithm. 

Definition 4.17. Let {Tl,Tr) be an A-labeled pair of twin binary search trees and a. be a 
letter. The insertion of a into (Tt , Tr) consists in making a leaf insertion of a into Tl and 
a root insertion of & into Tr. The iterative Baxter P-symbol (or simply iterative P-symbol 
if the context is clear) of a word u £ A* is the pair P{u) = (Tl,Tr) computed by iteratively 
inserting the letters of u, from left to right, into (_L,_L). The iterative Baxter Q-symbol (or 
simply iterative Q-symbol if the context is clear) of u £ A* is the pair Q(u) = (Sl,Sr) of same 
shape as P(u) and such that each node is labeled by its date of creation in P(u). 

Figure 11 shows, step by step, the computation of the iterative Baxter P and Q-symbols of 
a word. 

4.4.3. Correctness of the iterative insertion algorithm. To show that the iterative version of the 
Baxter P-symbol computes the same labeled pair of twin binary trees than its non-iterative 
version, we need the following lemma. 

Lemma 4.18. Let u £ A*. Let T be the right binary search tree obtained by root insertions 
of the letters of u, from left to right. Let T' be the right binary search tree obtained by leaf 
insertions of the letters of u, from right to left. Then, T = T' . 

Proof. Let us proceed by induction on \u\. If u = e, the lemma is satisfied. Otherwise, assume 
that u = v& where a G A. Let S be the right binary search tree obtained by root insertions of the 
letters of v from left to right. By induction hypothesis, S also is the right binary tree obtained 
by leaf insertions of the letters of v from right to left. The right binary search tree T obtained 
by root insertions of u from left to right satisfies, by definition, T = 5^ a A a S> a . The right 
binary search tree T" obtained by leaf insertions of u from right to left satisfies T' = L' A a R' 
where the subtree L' only depends on the subword w^ a := i>i]_oo,a] an d the subtree R' only 
depends on the subword t>> a := Wi] a ,+oo[; so that, by induction hypothesis, L' = S^, R' = S >SL 



Proposition 4.19. For any u £ A* , the Baxter P-symbol of u and the iterative Baxter P-symbol 
of u are equal. 

Proof. Let (Tl,Tr) be the P-symbol of u and (T' L ,T' R ) be the iterative P-symbol of u. By 
definition of these two insertion algorithms, we have Tl = T' L . Moreover, Tr is obtained by 
leaf insertions of the letters of u from right to left and T' R is obtained by root insertions of the 
letters of u from left to right. By Lemma 4.18, we have Tr = T' R . □ 




Figure 10. A right binary search tree T, T^ 2 and T >2 . 



and thus, T = T . 



□ 
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Figure 11. Steps of the computation of the P-symbol and the Q-symbol 
of u := 2415253. 



The correctness of the iterative version of the Q-symbol algorithm comes from the correctness 
of the iterative P-algorithm. 

5. The Baxter lattice 

5.1. The Baxter lattice congruence. Recall that an equivalence relation = on the elements 
of a lattice (L,A, V) is a lattice congruence if for all x,x',y,y' £ L, x = x' and y = y' imply 
x A y = x' A y' and x V y = x' V y'. The quotient L/= of L by = is naturally a lattice. Indeed, 
by denoting by r : L — > L/= the canonical projection, the set L/= is endowed with meet and 
join operations defined by x A y := t(x A y) and x V y := t(x V y) for all L/= where x 

and y are any elements of L such that r(a;) = x and r(y) = 

Lattices congruences admit the following very useful order-theoretic characterization [CS98, 
Rea05]. An equivalence relation = on the elements of a lattice (L, A, V) seen as a poset (L, ^) 
is a lattice congruence is the following three conditions hold. 

(LI) Every =-equivalence class is an interval of L; 

(L2) For any x, y G L, if x ^ y then x\, ^ yl where x\, is the maximal element of the 
=-equivalence class of x; 
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(L3) For any x, y £ L, if x ^ y then xf ^ y\ where x\ is the minimal element of the 
=-equivalence class of x. 

For any permutation a, let us denote by oj, (resp. of) the maximal (resp. minimal) permu- 
tation of the =b -equivalence class of a for the permutohedron order. Note by Proposition 3.8 
that oj, and a\ are well-defined. 

Theorem 5.1. The Baxter equivalence relation is a lattice congruence of the permutohedron. 

Proof. By Proposition 3.8, any Baxter equivalence class of permutations is an interval of the 
permutohedron, so that (LI) checks out. One just has to show that =b satisfies (L2) and (L3). 

Let a and v two permutations such that a v. Let us show that oj. v\.. It is enough 
to check the property when v = asi where Sj is an elementary transposition and i is not a 
descent of a. If a = oj., then oj- v ^p u\. and the property holds. Otherwise, by Lemma 3.9, 
there exists an elementary transposition Sj and a permutation tt such that tt and a are - 
adjacent, tt — crsj and a ^p tt. It then remains to prove that there exists a permutation /j, such 
that v =b [i and tt ^5p /i. Indeed, this leads to show, by applying iteratively this reasoning, 
that oj, is smaller than a permutation belonging to the =b -equivalence class of v for the 
permutohedron order and hence, by transitivity, that oj. ^p u\,. We have four cases: 

Case 1: If j ^ i — 2, a is of the form a = uabvcdw where it, u, and w are some words and a 
(resp. c) is the j-th (resp. i-th) letter of a. One has a < b and c < d since i and j are not 
descents of a. We have v = uabvdcw and vsj — abawdew =: fi. Moreover, since tt <=^b ct, 
there are some letters x 6 Alph(u) and y 6 Alph(u cdw) such that a < x, y < b. Thus, /i <=^b v. 
Finally, since tt = wbawcdw, tt ^p /i, so that /i is appropriate. 
Case 2: If j ^ i + 2, this is analogous to the previous case. 

Case 3: If j = i + 1, ct is of the form a = u abc v where u and v are some words and a is the 
i-th letter of a. One has a < b < c since i and j are not descents of a. Since a <=^b t 1 ", there 
are some letters x S Alph(w) and y G Alph(i>) such that b < x,y < c. Thus, since v — itbacv 
and a < b < x,y < c, we have vsj — wbcau^B^- Moreover, vsjSi — ucbaw =: fj, and 
vsj <=^b ^SjSi since b < x,y < c and thus, fi =b v. Finally, since tt = wacbu, we have tt ^p n, 
and hence is appropriate. 

Case 4: If j = i — 1, this is analogous to the previous case. 

Hence, the Baxter equivalence relation satisfies (L2). The proof that =b satisfies (L3) is 
analogous. □ 

5.2. A lattice structure over the set of pairs of twin binary trees. Recall that by 
Theorem 4.13, the Baxter equivalence classes of permutations are in correspondence with un- 
labeled pairs of twin binary trees. Thus, the quotient of the permutohedron of order n by the 
Baxter congruence is a lattice (TBT n , ^5b ) where the Baxter order relation ^5b satisfies, for 
any J , Ji £ TBT n , 

...... there are <r, v e & n such that 

(5.1) J s^b J\ if and only if 

a <p v, P (ct) = J and P (*/) = Ji. 

Let us call Baxter lattice the lattice (TBT n , ^b )■ Figure 12 shows the =b -equivalence classes 
in the permutohedron of order 4 that form the Baxter lattice (TBTa, ^b )• 

5.3. Covering relations of the Baxter lattice. Let us describe the covering relations of the 
lattice {TBTn , ^b ) in terms of operations on pairs of twin binary trees. Consider a Baxter 
equivalence class a of permutations encoded by a pair of twin binary trees (Tl,Tr). Let a 
by the maximal element of a. If i is a descent of er, the permutation erst is not in a, and, by 
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Figure 12. The permutohedron of order 4 cut into Baxter equivalence classes. 

definition of the Baxter lattice, the pair of twin binary trees P(asi) —: {T' L ,T' R ) covers (Tl,Tr). 
The permutations a and ast satisfy 

(5.2) <7 = uadu and asi = udav, 

where a < d. There are three cases whether the factor u or v contains a letter b satisfying 
a < b < d. Since the quotient of the permutohedron by the Sylvester congruence is the Tamari 
lattice [HNT05] and that covering relations in the Tamari lattice are binary tree rotations, the 
covering relations of the Baxter lattice are the following: 

(CI) If there is a letter b in v such that a < b < d, then T' R — Tjj and T' L is obtained from Tj, 

by performing a left rotation that does not change its canopy; 
(C2) If there is a letter b in u such that a < b < d, then T' L = Tl and T' R is obtained from Tr 

by performing a right rotation that does not change its canopy; 
(C3) If for any letter b of u and v, one has b < a or d < b, then T' L (resp. T' R ) is obtained 
from Tr, (resp. Tr) by performing a left (resp. right) rotation that changes its canopy. 
Hence, according to this characterization of the covering relations of the Baxter lattice 
and the definition of the Tamari lattice, we have, for any pairs of twin binary trees (Tl,Tr) 
and (T' L ,T R ), 

(5.3) (T L , T R ) < b {T' l , T' r ) if and only if T' L < T T L and T R ^ T T' R , 

Note that a right rotation at root y in a binary tree T changes its canopy if and only if the 
right subtree B of the left child x of y is empty (see Figure 1). Similarly, a left rotation at 
root y changes the canopy of T if and only if the left subtree B of y is empty. Moreover, if y 
is the i-th node of T, by Lemma 4.3, one can see that B is the i-th leaf of T. Hence, the right 
(resp. left) rotation at root y changes the orientation of the i-th leaf of T formerly on the right 
to the left (resp. left to the right). 

5.4. Twin Tamari diagrams. The purpose of this section is to introduce twin Tamari dia- 
grams. These diagrams are in bijection with pairs of twin binary trees and provide a useful 
realization of the Baxter lattice since it appears that testing if two twin Tamari diagrams are 
comparable under the Baxter order relation is immediate. 
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5.4.1. Tamari diagrams and the Tamari order relation. Pallo introduced in [Pal86] words in 
bijection with binary trees (see also [Knu06]). We call Tamari diagrams these words and to 
compute the Tamari diagram td(T) of a binary tree T , just label each node x of T by the 
number of nodes in the right subtree of x and then, consider its inorder reading. 

Any Tamari diagram S of length n satisfies the following two inequalities: 

(1) < Si < n - i, for all 1 i ^ n; 

(2) 5i+j ^ Si — j, for all 1 ^ i ^ n and 1 ^ j ^ Si. 

The main interest of Tamari diagrams is that they offer a very simple way to test if two 
binary trees are comparable in the Tamari lattice [Knu06]. Indeed, if T and T" are two binary 
trees with n nodes, one has 

(5.4) T < T T 1 if and only if td(T)j s$ td(T')i for all 1 < i < n. 

5.4.2. Twin Tamari diagrams and the Baxter order relation. 

Definition 5.2. A twin Tamari diagram of size n is a pair (S L ,S R ) such that S L and S R are 
Tamari diagrams of length n and for all index 1 ^ i ^ n — 1, exactly one letter among Sf 
and S R is zero. 

Note that we can represent any twin Tamari diagram 5 :— (S L , S R ^ in a more compact way 
by a word u>(5) were 



(5.5) w(S)t 



-Sf ifSf^O, 
5 R otherwise, 



for all 1 ^ i ^ n where n is the size of 5. We graphically represent a twin Tamari diagram S 
by drawing for each index i a column of |w(<$)j| boxes facing up if u>(5)i and facing down 
otherwise. First twin Tamari diagrams are drawn in Figure 13. 



-, nr, n_, 



EF- ^ dF-. "h-. ^- 



Figure 13. First twin Tamari diagrams of size 0, 1, 2, and 3. 



Proposition 5.3. For any n 0, the set of twin Tamari diagrams of size n is in bijection 
with the set of pairs of twin binary trees with n nodes. Moreover, this bijection is expressed as 
follows: If J := (Tl,Tr) is a pair of twin binary trees, the twin Tamari diagram in bijection 
with J is ttd(J) := (td{T L ),td(T R )). 

Proof. Let us show that the application ttd is well-defined, that is ttd(J) =: (8 L , S R ) is a twin 
Tamari diagram. Fix an index 1 ^ i ^ n — 1. By contradiction, assume first that Sf = S R = 0. 
By definition of td, this implies that the i-th nodes of Tl and Tr have no right child. Hence, by 
Lemma 4.3, the i+l-st leaves of Tl and Tr are attached to its i-th nodes and are right-oriented. 
Since i ^ n — 1, these leaves are not the rightmost leaves of Tl and Tr, implying that Tl and Tr 
have not complementary canopies, and hence that (Tl,Tr) is not a pair of twin binary trees. 
Assume now that Sf ^ and Sf ^ 0. By definition of td, this implies that the i-th nodes of Tl 
and Tr have a right child. Hence, by Lemma 4.3, the i+l-st leaves of Tl and Tr are attached 
to its i+l-st nodes and are left-oriented. This implies again that (Tl,Tr) is not a pair of twin 
binary trees. Thus, ttd computes twin Tamari diagrams. 
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Now, since td is a bijection between the set of binary trees with n nodes and Tamari diagrams 
of size n [Pal86], for any twin Tamari diagram 5, there is a unique pair of binary trees J such 
that ttd(J) = (5. Using very similar arguments as above, one can prove that the canopies of the 
trees of J are complementary, and hence, that J is a pair of twin binary trees. □ 

Figure 14 shows an example of a pair of twin binary trees with the corresponding twin Tamari 
diagram. 





ttd > (01041010,40100200) 

















+ 




J 



Figure 14. A pair of twin binary trees, the corresponding twin Tamari dia- 
gram via the bijection ttd and its graphical representation. 



Proposition 5.4. Let Jo and J\ two pairs of twin binary trees with n nodes. We have 

(5.6) Jo Jl if and only if lj (ttd ( Jo))^ ^ ^ (ttd ( Ji)) i for all 1 ^ i ^ n. 

Proof. This result is a direct consequence of the characterization of the Baxter order rela- 
tion (5.3) using the Tamari order relation, the characterization furnished by (5.4) to compare 
two binary trees in the Tamari lattice with Tamari diagrams, and the bijection between pairs 
of twin binary trees and Twin Tamari diagrams provided by Proposition 5.3. □ 

Figure 15 shows an interval of the Baxter lattice. 



LP 



P^ 



Figure 15. An interval of the Baxter lattice of order 5 where vertices are seen 
as pairs of twin binary trees and as Twin Tamari diagrams. 



6. The Hopf algebra of pairs of twin binary trees 
In the sequel, all the algebraic structures have a field of characteristic zero K as ground field. 
6.1. The Hopf algebra FQSym and construction of Hopf subalgebras. 
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6.1.1. The Hopf algebra FQSym. Recall that the family {Fo-j^^g forms the fundamental basis 
of FQSym, the Hopf algebra of Free quasi-symmetric functions [MR95, DHT02]. Its product 
and its coproduct are defined by 

(6.1) F ff -F„:= F -' 

7T £ a CD v 



(6.2) A(F CT ) := F std(«)®F 

For example, 



std(u) 

a—uv 



(6.3) 



(6.4) 



F132 • F12 = F13245 + F13425 + F13452 + F14325 + F14352 
+ F14532 + F41325 + F41352 + F41532 + F45132, 

A (F 3 5i 42 ) = 1 ® F 3 5i 42 + Fi ® F4132 + F12 ® Fi 32 

+ F 23 i ® F 2 i + F2413 ® Fi + F35142 <g> 1. 



Set G CT := F CT -i. Recall that FQSym is isomorphic to its dual FQSym* through the map 
ip : FQSym ->■ FQSym* defined by V> (Fa) := F*_ x = G* . 

Recall also that FQSym admits a polynomial realization [DHT02], that is an injective algebra 
morphism r A : FQSym M> K(A). Furthermore, this map should be compatible with the 
coalgebra structure in the sense that the coproduct of an element can be computed by taking 
its image by ta, and then by applying the alphabet doubling trick [DHT02, Hiv07]. This map 
is defined by 

(6.5) r A {G a ):= £ u. 

u e A* 

std(ti)— a 

For example, 

(6.6) r A (G € ) = l, 

(6.7) t a (Gi) = ^ a i = fl i + a 2 + a 3 H 

i 

(6.8) ta (G231) = a%ajau = 020201 + 020301 + 020401 ^ . 

6.1.2. Construction of Hopf subalgebras of FQSym. If = is an equivalence relation on © 
and a € 6, let us denote by a the =-equivalence class of a. 

The following theorem contained in an unpublished note of Hivert and Nzeutchap [HN07] 
(see also [DHT02,Hiv07]) shows that an equivalence relation on A* satisfying some properties 
can be used to define Hopf subalgebras of FQSym: 

Theorem 6.1. Let = be an equivalence relation defined on A* . If = is a congruence, compatible 
with the restriction of alphabet intervals and compatible with the destandardization process, then 
the family {Ps}a e e/ = defined by 

(6.9) P s := F - 

V E O 

spans a Hopf subalgebra of FQSym. 
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The compatibility with the destandardization process and with the restriction of alphabet 
intervals imply that for any appearing in a product P^-Pp and any permutation tt' = n, F w / 
also appears in the product. Moreover, the compatibility with the destandardization process 
and the fact that = is a congruence imply that for any F^^F^ appearing in a coproduct A (P5?) 
and any permutations a' = a and v' = v, F a i ® F„/ also appears in the coproduct. 

In the sequel, we shall call {Ps}s e @/ = the fundamental basis of the corresponding Hopf 
subalgebra of FQSym. 

6.2. Construction of the Hopf algebra Baxter. By Theorem 4.13, the =b -equivalence 
classes of permutations can be encoded by unlabeled pairs of twin binary trees. Moreover, in 
the sequel, the P-symbols of permutations are regarded as unlabeled pairs of twin binary trees 
since there is only one way to label a pair of twin binary trees with a permutation so that it is a 
pair of twin binary search trees. Hence, in our graphical representations we will only represent 
their shape. 

Since by definition =b is a congruence, since by Propositions 3.2 and 3.3, =b satisfies the 
conditions of Theorem 6.1, and since by Theorem 4.6, the permutations a such that P(cr) = J 
coincide with the Baxter equivalence class represented by the pair of twin binary trees J, we 
have the following theorem. 

Theorem 6.2. The family {P,/} j e fBT defined by 

(6.10) Pj:= F - 

a G 6 
P(<x) = J 

spans a Hopf subalgebra of FQSym, namely the Hopf algebra Baxter. 
For example, 

(6.11) Po ocf -F 12 , 

(6-12) P a o^ c ^a 3 = F2143 + F2413, 

(6.13) P jP 00 £ > --~ a ~ ^542163 + F5426I3 + F546213- 

The Hilbert series of Baxter is 

(6.14) B(z) :=l + z + 2z 2 + 6z 3 + 22z i + 92z 5 + 422z 6 + 2074z 7 + 10754z 8 + 58202 2 9 + ■ ■ • , 

the generating series of Baxter permutations (sequence A001181 of [Slo]). 

By Theorem 6.1, the product of Baxter is well-defined. We deduce it from the product 
of FQSym, and, since by Theorem 4.14 there is exactly one Baxter permutation in any =b - 
equivalence class of permutations, we obtain 

(6-15) Pjo-Pj^ E P PW- 

P(<r)=Jo, P(V)=Ji 
7r S a CD v n S B 
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For example, 

In the same way, we deduce the coproduct of Baxter from the coproduct of FQSym and 
by Theorem 4.14, we obtain 

(6.17) A(P, 7 )= J2 P P(CT )®Pp M . 

uv £ © 
P(uv)=.J 
<r:=std(ii), i/:=std(» 6 6 B 

For example, 
(6.18) 



A I P I = 1 ® + Poo ® Po^ ^ + Poo ^ P^ q% + Po_o ® Po. 



>cr Do 

,o ®Pnn +P„ o®P 00 + P CX.®PoO + P^^®l. 

D ° D cToSd cP 



6.3. Properties of the Hopf algebra Baxter. 

6.3.1. A polynomial realization. We deduce a polynomial realization of Baxter from the one 
of FQSym. In this section, we shall use the notation Jo ~ J\ to say that the labeled pairs of 
twin binary trees Jo and J± have same shape. 

Theorem 6.3. The map r A '■ Baxter — > K(A) defined by 



(6.19) r A (Pj) := £ 



u, 



u e A* 

(incr(u). dccr(u))~J 

for any J G TBT provides a polynomial realization of Baxter. 

Proof. Let us apply the polynomial realization rA of FQSym defined in (6.5) on elements of 
the fundamental basis of Baxter: 

(6.20) r A (P, } )= r ^( F -)' 

a e 6 
P(a)=J 



5-21) = r ^ G -)> 



a G 6 
P(<r- 1 ) = J 



3.22) = r A (G a ), 



a e 6 
(incr(er), dccr(CT))~J 



3.23) £ E ^ 



cr £ 6 u £ A" 

(incr(o-), decr(er))~J std(u)=CT 

The equality between (6.21) and (6.22) follows from Lemma 4.7. The equality between (6.23) 
and the right member of (6.19) follows from the fact that incr(cr) ~ incr(u) (resp. decr(c) ~ 
decr(u)) whenever std(u) = a. □ 
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6.3.2. The dual Hopf algebra. We denote by {P*,} /e7 - e7 - the dual basis of the basis {P j} j^fBT- 
The Hopf algebra Baxter*, dual of Baxter, is a quotient Hopf algebra of FQSym*. More 
precisely, 

(6.24) Baxter* = FQSym*//, 

where I is the Hopf ideal of FQSym* spanned by the elements (F* — F*) whenever a =b v. 

Let <fi : FQSym* -» Baxter* be the canonical projection, mapping F* on Pp( a y By 
definition, the product of Baxter* is 

(6.25) Pj„-n =<HK-K), 

where a and v are any permutations such that P(u) = Jo and P{v) = J\. Note that due to 
the fact that Baxter* is a quotient of FQSym*, the number of terms occurring in a product 
P* Jo ■ P*^ only depends on the number m (resp. n) of nodes of Jo (resp. Ji) and is ( m J^™)- For 
example, 

(6.26) 



P*^o -PX n = P* ^ + P Vc> r» + PVo r> + P* _o + P*o 

















+ p* 


-o 

3 dV 3 





, o 



o 



In the same way, the coproduct of Baxter* is 

(6.27) A(P. 7 H(0®0)(A(F*)), 

where a is any permutation such that P(c) = J. Note that the number of terms occurring in a 
coproduct A (P j) only depends on the number n of nodes of each binary trees of J and is n + 1. 
For example, 

(6.28) V ^cP°^° >cPo * ° 

+ P* n®P*, + P!p o ®1- 



Following Fomin [Fom94] (see also [BLL08]), we can build a pair of graded graphs in duality 
(Gp,Gp*). The set of vertices of Gp and Gp* is the set of pairs of twin binary trees. There 
is an edge between the vertices J and J' in Gp (resp. in Gp*) if Pji (resp. Pjv) appears in 
the product Pj ■ P 00 (resp. in the product P} • Pq ). Figure 16 (resp. Figure 17) shows the 
graded graph Gp (resp. Gp*) restricted to vertices of order smaller than 5. 

6.3.3. A boolean basis. We shall call a basis of an algebra (resp. coalgebra) a boolean algebra 
basis (resp. boolean coalgebra basis) if each element of the basis (resp. tensor square of the 
basis) only occurs with coefficient or 1 in any product (resp. coproduct) involving two (resp. 
one) elements of the basis. 

Proposition 6.4. If = is an equivalence relation defined on A* satisfying the conditions of 
Theorem 6.1 and additionally, for all n,fJ, G &, 

(6.29) cr, v £ 7T LU /i and cr^ 1 = v~ x imply a = v, 

then, the family {Ps-lage^ defined in (6.9) is both an algebra and a coalgebra boolean basis of 
the corresponding Hopf subalgebra of FQSym. 




Figure 16. The graded graph Gp restricted to vertices of order smaller than 5. 
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Figure 17. The graded graph Gp* restricted to vertices of order smaller than 5. 
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Proof. It is immediate from the definition of the product of FQSym that {Pa}aee/= 1S a 
boolean algebra basis, regardless of (6.29). 

By duality, {P?}$ e ©/ = 1S a boolean coalgebra basis if and only if its dual basis {P~} ffe6 ^ 
is a boolean algebra basis. One has 

(6.30) Pi-Pi = 0(p;-P*) 

(6.31) =^^(^- 1 (K)^- 1 (K))) 

(6.32) =^(^(F w -i.F M -i)) 
(6-33) = £ *(FS-0, 

(T £ 7T _1 CD I/ -1 

where is the canonical projection mapping F* on Pi for any permutation a, i/j is the Hopf 
isomorphism mapping F ff on F*_j for any permutation a, and tt € n and fj, £ fx. One can 
easily see that if = satisfies the hypothesis of the proposition, then there are no multiplicities 
in (6.33). □ 

Law and Reading have proved in [LR12] that the basis of their Baxter Hopf algebra, analog 
to our basis {Pj} Jg7 - Br , is both a boolean algebra basis and a boolean coalgebra basis. We 
re-prove this result in our setting: 

Proposition 6.5. The basis {P j} j^-j-BT * s a boolean algebra basis and a boolean coalgebra 
basis of Baxter. 

Proof. Let us prove that the Sylvester equivalence relation satisfies the assumptions of Propo- 
sition 6.4. Indeed, the result directly follows from the fact that, by Proposition 3.7, the Baxter 
equivalence relation is finer than the Sylvester equivalence relation. 

Let us start with a useful result: Let x and y be two words without repetition of same 
length and u,v G xlUy (here, the letters of y are shifted by max(i)). Let us prove by induction 
on \x\ + \y\ that if decr(it) and decr(v) have same shape, then u = v. It is obvious if |x| + \y\ = 0. 
Otherwise, one has u = u'bu" and v = v'bv" where b := max(w) = max(u). Since the shape 
of the left subtree of decr(u) is equal to the shape of the left subtree of decr(v), the position 
of b in u and v is the same. Moreover, the word y is of the form y = y' a y" where a := max(y), 
and x is of the form x — x'x", where u',v' £ x' LU y' and u",v" £ x" LU y". Since the left 
(resp. right) subtree of decr(w) is equal to the left (resp. right) subtree of decr(w), by induction 
hypothesis, u' — v' and u" — v" , showing that u = v. 

Now, let 7T, G & and a ^ v G 7r LU \x and assume that er -1 =g v~ x . Then, by Theorem 3.5, 
the permutations c -1 and v~ x give the same right binary search tree when inserted from 
right to left. By Lemma 4.7, that implies that decr(er) and decr(i/) have same shape. That 
implies a — is, contradicting our hypothesis. □ 

By duality, Proposition 6.5 also shows that the basis {Pj} /e7 - g7 - is a boolean algebra and 
coalgebra basis. 



6.3.4. A lattice interval description of the product. If = is an equivalence relation of © and a 
a permutation, denote by erf (resp. aj,) the minimal (resp. maximal) permutation of the =- 
equivalence class of a for the permutohedron order. 

Proposition 6.6. If = is an equivalence relation defined on A* satisfying the conditions of 
Theorem 6.1 and additionally, the =-equivalence classes of permutations are intervals of the 
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permutohedron, then the product on the family defined in (6.9) can be expressed as: 
(6.34) Ps-Pi7= J2 P - 

St/9|"<P7T <pS|.\K|. 
7T— min 7T 

Proof. It is well-known that the shifted shuffle product of two permutohedron intervals is still 
a permutohedron interval. Restating this fact in FQSym, we have 

(6-35) ( £ f]J £ F ) = E F,. 

By (6.35) and since that every =-equivalence class is an interval of the permutohedron, we 
obtain 

(6.36) P?-Pp= F - 

St/ £^<p t <p ct4.\S1, 

By Theorem 6.1, the expression (6.36) can be expressed as a sum of Pj elements and the 
proposition follows. □ 

Let Jo :— {T^,T R ) and J\ :— {Tl,T R ) be two pairs of twin binary trees. Let us define the 
pair of twin binary trees Jo / J\ by 

(6-37) Jo/ Ji := (T° L \Tl T^/T^). 

In the same way, the pair of twin binary trees Jo \ J\ is defined by 

(6.38) Jo \ Ji := (T° /T[, T° R \ T l R ) . 

Proposition 6.6 leads to the following expression for the product of Baxter. 
Corollary 6.7. For all pairs of twin binary trees Jq and Ji, the product of Baxter satisfies 



(6-39) Pj -P.h= E P 



j- 



Jo/ Jl <B J Jo\ Jl 

Proof. Let a and v two permutations. It is immediate, from the definition of the P-symbol 
algorithm, that the P-symbol of the permutation a / v (resp. o~\v) is the pair of twin binary 
trees P(er) /P(f) (resp. P(cr)\P(^)). The expression (6.39) follows from the fact that =b - 
equivalence classes of permutations are intervals of the permutohedron (Proposition 3.8) and 
from Proposition 6.6. □ 

6.3.5. Multiplicative bases and free generators. Recall that the elementary family {E°"} ctGS and 
the homogeneous family {H CT } CTee of FQSym respectively defined by 

cr' ) 



(6.40) E ff := £ F 

(6.41) H CT := F -'< 



form multiplicative bases of FQSym (see [AS05, DHNT11] for an exposition of some known 
bases of FQSym). Indeed, for all a, v £ 6, the product satisfies 

(6.42) E' T -E" = E CT/l/ , 

(6.43) IT W = H ffXl/ . 
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Mimicking these definitions, let us define the elementary family {^j} j^q-gj- and the homo- 
geneous family {Hj}j g1 - g1 - of Baxter respectively by 

(6.44) Ej:= P J> 

J J' 

(6.45) H, 7 := £ Pj,. 

J' (bJ 

These families are bases of Baxter since they are defined by triangularity. 

Proposition 6.8. Let J be a pair of twin binary trees and a\ (resp. o~\.) be the minimal (resp. 
maximal) permutation such that P(ot) = J (resp. P(aJ,) = J). Then, 

(6.46) E, 7 = E" 1 ", 

(6.47) H,/ = H^. 

Proof. Using the fact that, by Theorem 5.1, the =b -equivalence relation is a lattice congruence 
of the permutohedron, one successively has 

(6.48) E, 7 = J2 P ' = E E F ^ = E F ^= E f*=e*. 

P(i/)=J' ./^bPO) ot^p^ 

The proof for the homogeneous family is analogous. □ 
Corollary 6.9. For all pairs of twin binary trees Jq and J%, we have 
(6-49) E Jo -E, 7l =E Jo/Jl , 

(6-50) H. /0 .H 7l =H. 7oNJl . 

Proof. Let a and v be the minimal permutations of the =b -equivalence classes respectively 
encoded by Jo and J\. By Proposition 6.8, we have 

(6.51) E, 7o .E, 7l = E CT -E" = W /v . 

The permutation a / v is obviously the minimal element of its =b -equivalence class, and, by 
the definition of the P-symbol algorithm, the P-symbol of a / v is the pair of twin binary trees 
P(cr) / P(v) = Jo / J\. The proof of the second part of the proposition is analogous. □ 

For example, 

(6-52) E ^^-Eo^ ^o = E 



crocro o 




v u <^ - tr* 

Corollary 6.9 also shows that the {E,/} /e7 - B7 - and {H, 7 } /e7 - e7 - bases of Baxter are boolean 
algebra bases. However, these are not boolean coalgebra bases since one has 

(6.54) A(Eo oc p) = l®Ec bc p + 2Eoo®Eoo + Ec bc p® 1, 
and 

(6.55) A(HpcJ = l®H d 0Q o + 2Hoo®H oo + H o oQ o ® 1. 
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Let us say that a pair of twin binary trees J is connected (resp. anti- connected) if all the per- 
mutations a such that P(<x) = J are connected (resp. anti-connected). Since for any connected 
(resp. anti-connected) permutation a and a permutation v such that a v (resp. v a) the 
permutation v is also connected (resp. anti-connected), it is enough to check if the minimal 
(resp. maximal) permutation of the =b -equivalence class encoded by J is connected (resp. 
anti-connected) to decide if J is connected (resp. anti-connected). 

Lemma 6.10. For any pair of twin binary trees J , there exists a sequence of connected (resp. 
anti- connected) pairs of twin binary trees J\, . . . , J k such that 

(6.56) J=J X / ... /J k (resp. J = J x \ ■ ■ ■ \J k ). 

Proof. Let a be the minimal permutation of the =b -equivalence class encoded by J (recall 
that the existence of this element is ensured by Proposition 3.8). One can write a as 

(6.57) o = o-W/ ••• /(jW, 

where the permutations are connected for all 1 ^ i ^ k. Since a is the minimal permutation 
of its =b -equivalence class, all the permutations <rW are also minimal of their =b -equivalence 
classes. Hence, the pairs of twin binary trees P (crW) are connected and we can write 

(6.58) J=p(ct«) /•••/P(^ (fc) " 

The proof for the respective part is analogous. □ 

Theorem 6.11. The algebra Baxter is free on the elements Ej (resp. Hj) such that J is a 
connected (resp. anti- connected) pair of twin binary trees. 

Proof. By Corollary 6.9 and Lemma 6.10, each element Ej can be expressed as 

(6.59) Ej = E Jl .....E Jk , 

where the pairs of twin binary trees Ji are connected for all 1 ^ i ^ k. 

Now, since for all permutations a and v one has E"" • W — ~E a / v in FQSym, and since any 
permutation a admits a unique expression 

(6.60) a = a {1) / ■■■ / 

where , . . . , er^ are connected permutations, there is no relation in FQSym between the 
elements E CT where a is a connected permutation. 

Hence, by Proposition 6.8 and Corollary 6.9, there is also no relation in Baxter between the 
elements Ej where J is a connected pair of twin binary trees. The proof for the respective part 
is analogous. □ 

Let us denote by P>c(z) the generating series of connected (resp. anti-connected) pairs of 
twin binary trees. It follows, from Theorem 6.11, that the Hilbert series B(z) of Baxter satisfies 
B(z) = 1/ (1 — Bc(z)). Hence, the generating series Bc(z) satisfies 

(6.61) B c{z) = l- 1 ±- ) . 

First dimensions of algebraic generators of Baxter are 

(6.62) 0, 1, 1, 3, 11, 47, 221, 1113, 5903, 32607, 186143, 1092015. 
Here follows algebraic generators of Baxter of order 1 to 4: 

(6.63) E 00 ; 
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(6.64) E 



J Q 



o 



( 6 - 65 ) E n , En , E 



,o ( 



o 



f6661 \y E w W E « Vv E < 



o 



Proposition 6.12. If a is a connected (resp. anti- connected) Baxter permutation, then any 
permutation v such that a =b v is also connected (resp. anti- connected). 

Proof. As any permutation, every Baxter permutation a can be uniquely expressed as 

(6.67) a = a^/ ■ ■ ■ / a^ k \ 

where the permutations a^ 1 ' are connected for all 1 ^ i ^ k. Moreover, since a avoids the per- 
mutation patterns 2 — 41 — 3 and 3 — 14 — 2, the permutations also does, and hence, the <jW 
are Baxter permutations. This shows that the generating series of connected Baxter permuta- 
tions is Bc{z) and thus, that connected Baxter permutations, connected pairs of twin binary 
trees, and connected minimal permutations of Baxter equivalence classes are equinumerous. 

The proposition follows from Theorem 4.14 saying that each =b -equivalence class of per- 
mutations contains exactly one Baxter permutation. The proof for the respective part is anal- 
ogous. □ 

Corollary 6.13. The algebra Baxter is free on the elements Ej (resp. Hj) where the Baxter 
permutation belonging to the =b -equivalence class encoded by J is connected (resp. anti- 
connected). 

6.3.6. Bidendriform bialgebra structure and self-duality. A Hopf algebra (H, -,A) can be fit 
into a bidendriform bialgebra structure [Foi(J7] if (H + , -<,>-) is a dendriform algebra [LodOl] 
and (H + , A^, A>_) a codendriform coalgebra, where H + is the augmentation ideal of H. The 
operators -<, >-, A^ and A^ have to fulfill some compatibility relations. In particular, for 
all i,j £ H + , the product • of H is retrieved by x-y = x^y + x)^y and the coproduct A 
of H is retrieved by A(x) = 1 ® x + A^(x) + Ay(x) + x ® 1. Recall that an element x e H + is 
totally primitive if A_<(x) = = A>_(x). 

The Hopf algebra FQSym admits a bidendriform bialgebra structure [Foi07]. Indeed, for 
all <7, v £ <5„ with n 1, set 

(6.68) F CT ^F„:= £ F x , 

7T £ tr LU V 

7r|„|=o-|„| 

(6.69) F ff ^F„:= ^ P„ 

7T £ CT UJ V 

7r| w |=i/| v | + |tr| 

(6.70) A^(F CT ):= ^ F std(u) <g> F std(i;) , 

max(ii)— max(er) 
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(6.71) A^(F CT ) := ]T F s td(«) ® F st d(i;)- 

rr—uv 
max(u)— max(ff) 

Proposition 6.14. // = is an equivalence relation defined on A* satisfying the conditions of 
Theorem 6.1 and additionally, for all u,v £ A*, the relation u = v implies U\ u \ — Vi v i, then, the 
family defined in (6.9) spans a bidendriform sub-bialgebra o/FQSym that is free as an algebra, 
cofree as a coalgebra, self-dual, free as a dendriform algebra on its totally primitive elements, 
and the Lie algebra of its primitive elements is free. 

Proof. It is enough to show that the operators -<, >-, A^ and A^ of FQSym are well-defined 
in the Hopf subalgebra H of FQSym spanned by the elements {Ps}s e e/=- ^ n tms wa -y> H is 
endowed with a structure of bidendriform bialgebra and the results of Foissy [Foi07] imply the 
rest of the proposition. 

Fix J,? £ ©/= and an element F^ appearing in the product Ps -< Pp. Hence, there is 
a permutation a <= a such that m^i = o~\o\- Let w' a permutation such that n = ir' . By 
Theorem 6.1, the element appears in the product P^ • Pp, and hence, it also appears in 
Pg -< Pp or in Pff y Pp. Assume by contradiction that F n > appears in P^ >- Pp. There are 
two permutations a' £a and v' £ v such that 7rL.,i = + That implies that ir\„\ ^ 7rL, 
and contradicts the fact that all permutations of a same =-equivalence class end with a same 
letter. Hence, the element F T / appears in P^ ~< Pp, showing that the product -< is well-defined 
in H . Then so is >- since -< + >- is the whole product. 

Fix a G &/= and an element F„ (gi F T appearing in the coproduct Ax(Ps)- Hence, there is 
a permutation a £ a such that a = uv, v — std(w), 7r = std(w) and the maximal letter of uv is 
in the factor u. Now, let v' and n' be two permutations such that v = i/', 7r = 7r'. Let us show 
that the element F„/ ®F T / also appears in A^(Ps). For that, let u' be a permutation of it such 
that std(w') = u', and v' be a permutation of v such that std(w') = n'. Since ev(u') = ev(u), 
std(u') = std(w), and = is compatible with the destandardization process, one has u = v! . 
For the same reason, v = v' , and since = is a congruence, one has uv = u'v'. Finally, since 
the maximal letter of uv is in u, the maximal letter of u'v 1 is in u' , showing that the element 
F„/ ®F„i appears in A^(P^). Thus, the coproduct A^ is well-defined in H. The proof for the 
coproduct A^ is analogous. □ 



Corollary 6.15. The Hopf algebra Baxter is free as an algebra, cofree as a coalgebra, self- 
dual, free as a dendriform algebra on its totally primitive elements, and the Lie algebra of its 
primitive elements is free. 

Proof. Since all words of a same =b -equivalence class end with a same letter, =b satisfies the 
premises of Proposition 6.14 and hence, Baxter satisfies all stated properties. □ 



Considering the map 9' : PBT FQSym that is the injection from PBT to FQSym and 
0' : FQSym* -» PBT* the surjection from FQSym* to PBT*, it is well-known (see [HNT05]) 
that the map <f>' o if) o 8' induces an isomorphism between PBT and PBT*. Hence, since by 
Corollary 6.15, the Hopf algebras Baxter and Baxter* are isomorphic, it is natural to test if 
an analogous map is still an isomorphism between Baxter and Baxter*. However, denoting by 
9 : Baxter ^ FQSym the injection from Baxter to FQSym, the map cj> o ip o 9 : Baxter — > 



ALGEBRAIC AND COMBINATORIAL STRUCTURES ON PAIRS OF TWIN BINARY TREES 



39 



Baxter* is not an isomorphism. Indeed 
(6.72) 



o o 



<j> o $ o 0[ P^x o 0C a o )=<t>°i> (F2143 + F 24 i 3 ) = (F* 143 + F* 142 ) = P* % ^ + P; 
(6.73) 



^ ° ^ ° 9 I V r^ n cfSD J = ^ ° ^ ( F3142 + F3412 ) = ^ (F 2413 + F 341 2 ) = P* Q Q Q O q + 



showing that (j> o o Q is not injective. 

6.3.7. Primitive and totally primitive elements. Since the family {Ej} j£C (resp. {H,/}j eC ), 
where C is the set of connected (resp. anti-connected) pairs of twin binary trees are indecom- 
posable elements of Baxter, its dual family {Ej} JgC , (resp. {H*,} JgC ) forms a basis of the Lie 
algebra of the primitive elements of Baxter*. By Corollary 6.15, this Lie algebra is free. 

Following [Foi07], the generating series Bt(z) of the totally primitive elements of Baxter is 

First dimensions of totally primitive elements of Baxter are 

(6.75) 0, 1, 0, 1, 4, 19, 96, 511, 2832, 16215, 95374, 573837. 

Here follows a basis of the totally primitive elements of Baxter of order 1, 3 and 4: 

(6.76) *i,i=Poo, 



V^~ P >cPo ~ PC ^^V 

P o-po^- P c^cf>- 

6.3.8. Compatibility with the product. Aval and Viennot [AV10] endowed PBT with a new 
associative product called the # product. The product of two elements of PBT of degrees n 
and m is an element of degree n + m — 1. Aval, Novelli, and Thibon [ANT11] generalized 
the # product at the level of the associative algebra and showed that it is still well-defined 
in FQSym. 

Let for all k ^ 1 the linear maps dk : FQSym — > FQSym defined for any permutation a 
of & n by 
(6.82) 

, s J F8td(cTi...<r 4 <r 4+ 2...<r n ) if there is 1 < i < n - 1 such that o { = k and =k + l, 
otherwise. 



(6.77) i 3 ,i = 

(6.78) t iA = 

(6.79) < 4 ,2 = 

(6.80) h, 3 = 

(6.81) t 4 , 4 = 
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Now, for any permutations a and v, the ^-product is denned in FQSym by 
(6.83) F ff #F„ := d n (F CT • F„) , 

where n is the size of a. 

Proposition 6.16. The linear maps dk are well-defined in Baxter. More precisely, one has 
for any pair of twin binary trees J := (To, T\), 

!p if the k + l-st (resp. k-th) node is a child of the 
J ' k-th (resp. k+l-st) node in T L (resp. T R ), 
otherwise, 

where J' := {T' L ,T' R ) is the pair of twin binary trees obtained by contracting in Tl and T R the 
edges connecting the k-th and the k+l-st nodes. 

Proof. This proof relies on the fact that, according to Proposition 4.10, the permutations of a 
Baxter equivalence class coincide with linear extensions of the posets A(T^) and \/(T R ). 

We have two cases to consider whether the fc + Tst (resp. fc-th) node is a child of the fc-th 
(resp. fc+l-st) node in Tl (resp. T R ). 

Case 1. If so, there is in the Baxter equivalence class represented by J some permutations with 
a factor k.(k+l). The map dk deletes letters k+l in these permutations and standardizes them. 
The obtained permutations coincide with linear extensions of the posets A(T£J and \/(T' R ). 
Case 2. If this is not the case, since the fc-th and fc + l-st nodes of a binary tree are on a 
same path starting from the root, no permutation of the Baxter class represented by J has a 
factor fc.(fc+l). Hence, d k (Pj) = 0. □ 

One has for example 
(6.85) 

Proposition 6.16 shows in particular that the # product in well-defined in Baxter. 
6.4. Connections with other Hopf subalgebras of FQSym. 

6.4.1. Connection with the Hopf algebra PBT. We already recalled that the Sylvester con- 
gruence leads to the construction of the Hopf subalgebra PBT [LR98] of FQSym, whose 
fundamental basis 

(6.86) {P T : T <= BT} 

is defined in accordance with (6.9) (see [HNT02] and [HNT05]). By Proposition 3.7, every 
=S -equivalence class is a union of some =b -equivalence classes. Hence, we have the following 
injective Hopf map: 

(6.87) p : PBT ^ Baxter, 

satisfying 

(6-88) p(Pt)= E P ''' 




T' e BT 
J:=(T',T) G TBT 



for any binary tree T. For example, 
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6.4.2. Connection with the Hopf algebra DSym' 3 ). The congruence =r<3) leads to the con- 
struction of the Hopf subalgcbra DSym' 3 ) of FQSym, whose fundamental basis 

(6.90) {P ? ;5e6/, R(3) } 

is defined in accordance with (6.9) (see [NRTT1]). By Proposition 3.10, every = R <3> -equivalence 
class of permutations is a union of some =b -equivalence classes. Hence, we have the following 
injective Hopf map: 

(6.91) a : DSym (3) ^ Baxter, 

satisfying 

(6.92) a(P s )= *V)> 

a e Sn& B 

for any = R (3> -equivalence class a of permutations. 

6.4.3. Connection with the Hopf algebra Sym. The hypoplactic congruence [Nov98] leads to the 
construction of the Hopf subalgebra Sym of FQSym. As already mentioned, the hypoplactic 
congruence is the same as the congruence = R <2) when both are restricted on permutations. 
Moreover, the hypoplactic equivalence classes of permutations can be encoded by binary words. 
Indeed, if a is such an equivalence class, a contains all the permutations having a given recoil 
set. Thus, the class a can be encoded by the binary word b of length n — 1 where n is the length 
of the elements of a and bi = 1 if and only if i is a recoil of the elements of a. We denote by 

(6.93) {P b :&£{0,l}*} 

the fundamental basis of Sym indexed by binary words. 

Since PBT is a Hopf subalgebra of Baxter and Sym is a Hopf subalgebra of PBT [HNT05], 
Sym is itself a Hopf subalgebra of Baxter. The injective Hopf map 

(6.94) (3 : Sym ^ PBT, 

satisfies, thanks to the fact that the hypoplactic equivalence classes are union of =s -equivalence 
classes and Proposition 4.4, 

(6.95) /3(P h )= ]T Pt, 

T 6 BT 
cnp(T)=b 

for any binary word b. From a combinatorial point of view, given a binary word b, the map f3 
computes the sum of the binary trees having b as canopy. The composition p o f3 is an injective 
Hopf map from Sym to Baxter. From a combinatorial point of view, given a binary word 6, 
the map p o fj computes the sum of the pairs of twin binary trees (?l,Tr) where the canopy 
of Tr is b and the canopy of is the complementary of b. 

6.4.4. Full diagram of embeddings. Figure 18 summarizes the relations between known Hopf 
algebras related to Baxter. 
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